[Development] New Qt Multimedia

[Jason H]

> > 4. On the removal of QAbstractVideoFilter AND QVideoProbe: Disappointed to hear this. I previously used this for read-only frames for analysis, i.e. Barcode reading and object detection. How do we do that now?
> 
> You can get full access to the video data from a QVideoSink through the newVideoFrame() signal and mapping the video frame (or converting it to a QImage). With that you should have the access to the data that you need. (A small detail that’s still missing right now but that I hope to have in place for 6.2 is the ability to connect several sinks to a media player or capture session, but it’s not really required as you can connect several slots to the signal). If you feel like to need more than that, let me know what it is.

So the most common case I need supported is getting the video frame for processing while maintaining the full-speed live preview. Is this the multi-sink scenario?
Typically, With Camera connected o a VideoOutput, I use QVideoProbe to throw the frame (pixel data as QByteArray, because the library doesn't care) to a thread for multicore async processing. A typical 1 megapixel image on RaspberryPi4 takes ~150ms using ZBar or ZXing (I find ZXing is more like 100ms), so this gets about 6 processed frames a second, which seems responsive enough to the user because they are looking at the live display.

Since you asked for actual code, attached is the code I use to do this. It may not be perfect code (long story made short, I just rewrote this from memory) but it is what I whipped up, and works reasonably well for now.  I've used this approach for barcodes and OpenCV.

If it matters: I disclaim any copyright for the attached files.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: barcodevideoprobe.cpp
Type: application/octet-stream
Size: 4937 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/development/attachments/20210526/93469975/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: barcodevideoprobe.h
URL: <http://lists.qt-project.org/pipermail/development/attachments/20210526/93469975/attachment.h>