[Interest] Faster way of rendering QOpenGLFramebufferObject to image

[Elvis Stansvik]

My idea would be something like (sketch):

main.cpp:

#include <QGuiApplication>
#include <QQuickView>
#include <QObject>
#include <QMutex>
#include <QVector>

int main(int argc, char *argv[])
{
   QGuiApplication app(argc, argv);

   auto view = new QQuickView;
   view->setSource(QUrl::fromLocalFile("test.qml"));
   view->show();

   // I use a QVector here, but you would have your AVFrame and
   // its allocated memory buffer..
   QVector<unsigned char> buffer;
   QMutex mutex;

   QObject::connect(view, &QQuickView::afterRendering, view, [&]() {
       mutex.lock();
       buffer.resize(4 * view->width() * view->height());
       glReadPixels(0, 0, view->width(), view->height(), GL_RGBA,
GL_UNSIGNED_BYTE, buffer.data());
       mutex.unlock();
   }, Qt::DirectConnection);

   return app.exec();
}

test.qml:

import QtQuick 2.2

Rectangle {
   width: 1920
   height: 1080

   Rectangle {
       width: 500
       height: 500
       color: "green"
       anchors.centerIn: parent
       RotationAnimation on rotation {
           from: 0
           to: 360
           duration: 2000
           loops: Animation.Infinite
       }
   }
}

I don't know whether this would work for you.

Elvis

Den tis 13 juli 2021 kl 15:31 skrev Nuno Santos <nuno.santos at imaginando.pt>:
>
> Elvis,
>
> Thanks for sharing your thoughts. It makes sense.
>
> I need to dive into the toImage function, try to read directly the bytes from the FBO and see if that has any performance impact.
>
> Best regards,
>
> Nuno
>
> > On 13 Jul 2021, at 14:10, Elvis Stansvik <elvstone at gmail.com> wrote:
> >
> > Hi Nuno,
> >
> > I'm really out of my waters here, but, provided you don't need to hold
> > on to each AVFrame after you are "done with it", you could perhaps
> > avoid having to allocate a QImage for each frame (which toImage forces
> > you to do) by just allocating a a single AVFrame and a single memory
> > buffer for it, and then do what toImage does, which is make sure the
> > FBO is bound and read the pixels off of it with glReadPixels. Then you
> > could read the pixels straight into the memory buffer used by your
> > AVFrame.
> >
> > That way you would save the overhead of a new QImage being allocated
> > each time, which might speed things up..?
> >
> > Just ideas here. Have not worked with GL or ffmpeg before.
> >
> > Elvis
> >
> > Den tis 13 juli 2021 kl 11:22 skrev Nuno Santos <nuno.santos at imaginando.pt>:
> >>
> >> Hi,
> >>
> >> I’m trying to capture the content of an FBO to a video file. This video file should contain the animations generated by a qml scene.
> >>
> >> To do this, I’m recurring to QOpenGLFramebufferObject class toImage() method.
> >>
> >> My scene is being drawn at 1920x1080. Each call to toImage takes 30 ms! :(
> >>
> >> If I want to render to file at 60 fps, ideally, this call would need to take less than 16 ms to give me room to do other operations, such as video encoding and the actual render.
> >>
> >> As anyone been here before? What other strategies are available to copy the FBO data to an image?
> >>
> >> I’m using libav to encode the video file, therefor I need to fill an AVFrame. Right now I’m filling the AVFrame from the QImage generated by the FBO toImage method.
> >>
> >> Does any one knows a method of filling an AVFrame directly from texture data?
> >>
> >> Thanks
> >>
> >> Best regards,
> >>
> >> Nuno
> >> _______________________________________________
> >> Interest mailing list
> >> Interest at qt-project.org
> >> https://lists.qt-project.org/listinfo/interest
>