[Interest] Faster way of rendering QOpenGLFramebufferObject to image

[Elvis Stansvik]

Den tors 15 juli 2021 kl 10:30 skrev Nuno Santos <nuno.santos at imaginando.pt>:
>
> Elvis,
>
> Using your quick example (attached as a project below) I’ve run some tests and realised that window rendering it not happening at the same frame rate in all machines and the maximum differs:
>
> - no more than 30 fps in on a 2017 iMac with Radeon graphics card
> - no more than 30 fps on 2019 Intel Mac Book with Intel Graphics
> - up to 144 fps on a Windows machine with an RTX 2060
> - 115 fps on a 2018 12.9” iPad Pro
>
> Changing the width and height of the scene doesn’t seem to impact the frame rate.
>
> I was expecting to have more than 60 fps on the iMac and Macbook. I don’t think this simple scene would be limited to 30 fps

It does sound like a bug to me, unless somehow those Mac systems are
running at a 30 Hz monitor refresh rate and Quick is trying to match
that, but it seems very unlikely.

Unless someone here knows, I'd file a bug with a minimal example.

I did try adding some basic timing to my earlier example, and on my
Linux laptop afterRendering is called at a rate of 60 Hz.

Elvis

>
> At this point, I’m not taking into consideration that time it takes to obtain the pixel data from the fbo/texture.
>
> Since in order to have v-sync with window when painting the context of the texture rendered on the offscreen renderer I need to drive the paint via main window paint, this means that I will be limited by main window painting frame rate.
>
> Is anyone aware of frame rate limitations of a qquickwindow painting?
>
> Project attached.
>
> Thanks,
>
> Best regards,
>
> Nuno
>
>
>
> > On 14 Jul 2021, at 23:48, Elvis Stansvik <elvstone at gmail.com> wrote:
> >
> > My idea would be something like (sketch):
> >
> > main.cpp:
> >
> > #include <QGuiApplication>
> > #include <QQuickView>
> > #include <QObject>
> > #include <QMutex>
> > #include <QVector>
> >
> > int main(int argc, char *argv[])
> > {
> >   QGuiApplication app(argc, argv);
> >
> >   auto view = new QQuickView;
> >   view->setSource(QUrl::fromLocalFile("test.qml"));
> >   view->show();
> >
> >   // I use a QVector here, but you would have your AVFrame and
> >   // its allocated memory buffer..
> >   QVector<unsigned char> buffer;
> >   QMutex mutex;
> >
> >   QObject::connect(view, &QQuickView::afterRendering, view, [&]() {
> >       mutex.lock();
> >       buffer.resize(4 * view->width() * view->height());
> >       glReadPixels(0, 0, view->width(), view->height(), GL_RGBA,
> > GL_UNSIGNED_BYTE, buffer.data());
> >       mutex.unlock();
> >   }, Qt::DirectConnection);
> >
> >   return app.exec();
> > }
> >
> > test.qml:
> >
> > import QtQuick 2.2
> >
> > Rectangle {
> >   width: 1920
> >   height: 1080
> >
> >   Rectangle {
> >       width: 500
> >       height: 500
> >       color: "green"
> >       anchors.centerIn: parent
> >       RotationAnimation on rotation {
> >           from: 0
> >           to: 360
> >           duration: 2000
> >           loops: Animation.Infinite
> >       }
> >   }
> > }
> >
> > I don't know whether this would work for you.
> >
> > Elvis
> >
> > Den tis 13 juli 2021 kl 15:31 skrev Nuno Santos <nuno.santos at imaginando.pt>:
> >>
> >> Elvis,
> >>
> >> Thanks for sharing your thoughts. It makes sense.
> >>
> >> I need to dive into the toImage function, try to read directly the bytes from the FBO and see if that has any performance impact.
> >>
> >> Best regards,
> >>
> >> Nuno
> >>
> >>> On 13 Jul 2021, at 14:10, Elvis Stansvik <elvstone at gmail.com> wrote:
> >>>
> >>> Hi Nuno,
> >>>
> >>> I'm really out of my waters here, but, provided you don't need to hold
> >>> on to each AVFrame after you are "done with it", you could perhaps
> >>> avoid having to allocate a QImage for each frame (which toImage forces
> >>> you to do) by just allocating a a single AVFrame and a single memory
> >>> buffer for it, and then do what toImage does, which is make sure the
> >>> FBO is bound and read the pixels off of it with glReadPixels. Then you
> >>> could read the pixels straight into the memory buffer used by your
> >>> AVFrame.
> >>>
> >>> That way you would save the overhead of a new QImage being allocated
> >>> each time, which might speed things up..?
> >>>
> >>> Just ideas here. Have not worked with GL or ffmpeg before.
> >>>
> >>> Elvis
> >>>
> >>> Den tis 13 juli 2021 kl 11:22 skrev Nuno Santos <nuno.santos at imaginando.pt>:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I’m trying to capture the content of an FBO to a video file. This video file should contain the animations generated by a qml scene.
> >>>>
> >>>> To do this, I’m recurring to QOpenGLFramebufferObject class toImage() method.
> >>>>
> >>>> My scene is being drawn at 1920x1080. Each call to toImage takes 30 ms! :(
> >>>>
> >>>> If I want to render to file at 60 fps, ideally, this call would need to take less than 16 ms to give me room to do other operations, such as video encoding and the actual render.
> >>>>
> >>>> As anyone been here before? What other strategies are available to copy the FBO data to an image?
> >>>>
> >>>> I’m using libav to encode the video file, therefor I need to fill an AVFrame. Right now I’m filling the AVFrame from the QImage generated by the FBO toImage method.
> >>>>
> >>>> Does any one knows a method of filling an AVFrame directly from texture data?
> >>>>
> >>>> Thanks
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Nuno
> >>>> _______________________________________________
> >>>> Interest mailing list
> >>>> Interest at qt-project.org
> >>>> https://lists.qt-project.org/listinfo/interest
> >>
>