[Interest] Faster way of rendering QOpenGLFramebufferObject to image

[Nuno Santos]

Elvis,

Your sketch runs fine. 

Yesterday night I have been playing with calling glReadPixels directly rather than calling toImage from FBO object. Apparently, there wasn't much difference in the machine I was running this tests. I need to test on other machines.

But I also realised that the beforeRendering signal was by itself limited to 30 fps. I was doing this tests on a Retina 5K 27” Intel iMac. I’ve been doing tests on other machines like a Windows laptop with a powerful NVIDIA gpu, an Intel i5 quad code machine and an Apple M1 iPad. I don’t have precise numbers right now, but I’ve had the frame rate above 60, and mostly on the Windows machine, I’ve seen frame rate of 150 fps.

But for some reason, in this machine, even with your simple example, afterRendering is not called more than 30 fps.

This raises the question: who is throttling the QQuickView? And why this value is not constant between machines? In the mean time I will run the same example on all other machines to get more precise numbers.

Best regards,

Nuno

> On 14 Jul 2021, at 23:48, Elvis Stansvik <elvstone at gmail.com> wrote:
> 
> My idea would be something like (sketch):
> 
> main.cpp:
> 
> #include <QGuiApplication>
> #include <QQuickView>
> #include <QObject>
> #include <QMutex>
> #include <QVector>
> 
> int main(int argc, char *argv[])
> {
>   QGuiApplication app(argc, argv);
> 
>   auto view = new QQuickView;
>   view->setSource(QUrl::fromLocalFile("test.qml"));
>   view->show();
> 
>   // I use a QVector here, but you would have your AVFrame and
>   // its allocated memory buffer..
>   QVector<unsigned char> buffer;
>   QMutex mutex;
> 
>   QObject::connect(view, &QQuickView::afterRendering, view, [&]() {
>       mutex.lock();
>       buffer.resize(4 * view->width() * view->height());
>       glReadPixels(0, 0, view->width(), view->height(), GL_RGBA,
> GL_UNSIGNED_BYTE, buffer.data());
>       mutex.unlock();
>   }, Qt::DirectConnection);
> 
>   return app.exec();
> }
> 
> test.qml:
> 
> import QtQuick 2.2
> 
> Rectangle {
>   width: 1920
>   height: 1080
> 
>   Rectangle {
>       width: 500
>       height: 500
>       color: "green"
>       anchors.centerIn: parent
>       RotationAnimation on rotation {
>           from: 0
>           to: 360
>           duration: 2000
>           loops: Animation.Infinite
>       }
>   }
> }
> 
> I don't know whether this would work for you.
> 
> Elvis
> 
> Den tis 13 juli 2021 kl 15:31 skrev Nuno Santos <nuno.santos at imaginando.pt>:
>> 
>> Elvis,
>> 
>> Thanks for sharing your thoughts. It makes sense.
>> 
>> I need to dive into the toImage function, try to read directly the bytes from the FBO and see if that has any performance impact.
>> 
>> Best regards,
>> 
>> Nuno
>> 
>>> On 13 Jul 2021, at 14:10, Elvis Stansvik <elvstone at gmail.com> wrote:
>>> 
>>> Hi Nuno,
>>> 
>>> I'm really out of my waters here, but, provided you don't need to hold
>>> on to each AVFrame after you are "done with it", you could perhaps
>>> avoid having to allocate a QImage for each frame (which toImage forces
>>> you to do) by just allocating a a single AVFrame and a single memory
>>> buffer for it, and then do what toImage does, which is make sure the
>>> FBO is bound and read the pixels off of it with glReadPixels. Then you
>>> could read the pixels straight into the memory buffer used by your
>>> AVFrame.
>>> 
>>> That way you would save the overhead of a new QImage being allocated
>>> each time, which might speed things up..?
>>> 
>>> Just ideas here. Have not worked with GL or ffmpeg before.
>>> 
>>> Elvis
>>> 
>>> Den tis 13 juli 2021 kl 11:22 skrev Nuno Santos <nuno.santos at imaginando.pt>:
>>>> 
>>>> Hi,
>>>> 
>>>> I’m trying to capture the content of an FBO to a video file. This video file should contain the animations generated by a qml scene.
>>>> 
>>>> To do this, I’m recurring to QOpenGLFramebufferObject class toImage() method.
>>>> 
>>>> My scene is being drawn at 1920x1080. Each call to toImage takes 30 ms! :(
>>>> 
>>>> If I want to render to file at 60 fps, ideally, this call would need to take less than 16 ms to give me room to do other operations, such as video encoding and the actual render.
>>>> 
>>>> As anyone been here before? What other strategies are available to copy the FBO data to an image?
>>>> 
>>>> I’m using libav to encode the video file, therefor I need to fill an AVFrame. Right now I’m filling the AVFrame from the QImage generated by the FBO toImage method.
>>>> 
>>>> Does any one knows a method of filling an AVFrame directly from texture data?
>>>> 
>>>> Thanks
>>>> 
>>>> Best regards,
>>>> 
>>>> Nuno
>>>> _______________________________________________
>>>> Interest mailing list
>>>> Interest at qt-project.org
>>>> https://lists.qt-project.org/listinfo/interest
>>