[Interest] QMediaCaptureSession capture from QGraphicsScene?

[Volker Hilsheimer]

> On 19 Jun 2024, at 20:14, David M. Cotter <dave at kjams.com> wrote:
> 
> If i have a QGraphicsScene (not necessarily connected to a view, or if required, an OFFSCREEN view), can i set its output as the input to a QMediaCaptureSession?
> I see i can connect a QWindowCapture, but must that window be showing? can that window be invisible? eg: h.264 output in the background, unrelated to the showing (visible) windows?
> I had asked this question a couple years ago, but times have changed and i wonder if it is now possible?
> if it is not possible from QGraphicsScene directly, then how about a temporary buffer of QImage? Can i feed QMediaCaptureSession a QImageCapture class, then feed the QImageCapture class a series of QImage or QVideoFrame refs?
> is there sample code for sending in-memory images into QMediaCaptureSession ?
> 
> -Dave


Hi Dave,

Forwarding Artem’s reply (which went to development@):


Thanks for the questions! 

In 6.8, we added QVideoFramesInput, and QAudioBufferInput, which can be connected toQMediaCaptureSession as inputs.
Let's go through the questions.

>  If i have a QGraphicsScene (not necessarily connected to a view, or if required, an OFFSCREEN view), can i set its output as the input to a QMediaCaptureSession?

You cannot set QGraphicsScene as an input to QMediaCaptureSession. Instead, you may render QGraphicsScene to QImage, construct QVideoFrame from the image, and send your sequence of images to QVideoFrameInput, connected to QMediaCaptureSession.


> I see i can connect a QWindowCapture, but must that window be showing? can that window be invisible? eg: h.264 output in the background, unrelated to the showing (visible) windows?

It’s platform-dependent. E.g. on Windows, we cannot capture invisible windows. On other platforms, it’s to be double-checked and documented, the ticket is https://bugreports.qt.io/browse/QTBUG-126552. 
In general, the result of QWindowCapture::capturableWindows is designed to represent the windows we can capture. If the list contains an invisible window that QtMM cannot capture, a ticket might be reported.


> if it is not possible from QGraphicsScene directly, then how about a temporary buffer of QImage? Can i feed QMediaCaptureSession a QImageCapture class, then feed the QImageCapture class a series of QImage or QVideoFrame refs?

QImageCapture is rather an output for capturing image shots from cameras, screens, windows, so it doesn’t cover your needs. Please use QVideoFrameInput. In 6.8, you may create video frames from QImage or from your custom data via QAbstractVideoBuffer, see the constructors of QVideoFrame.


> is there sample code for sending in-memory images into QMediaCaptureSession ?


There are no code samples in the public documentation yet. We have a ticket for writing them:https://bugreports.qt.io/browse/QTBUG-126319.
Until we add the snippets or examples, you may take a look at the tests:
An early tests revision with a straight-forward flow.
The current tests revision is a bit harder to learn from. 



Regards
Artem