[Qt-interest] Using QuickTime with QGLWidget

[Nechypurenko, Andrey (GE Healthcare)]

 Hi, 

> I developed a Qt application on Mac OS X Snow Leopard. My main issue 
> was drawing video frames in 3D space(using OpenGL) realtime on Mac 
> environment.

Recently I submit the new example to Gstreamer which might be also of interest for you: 
https://bugzilla.gnome.org/show_bug.cgi?id=600797 (source code is attached to the report).

This example shows how to integrate Gstreamer's gst-plugins-gl and Qt. In particular it creates GL texture with video frame and then draws this texture on rotating cube using QGLWidget derived class. It works on Linux and Windows, so chances are pretty high that it will also works on Mac ;-) .


Regards,
Andrey.

> -----Original Message-----
> From: qt-interest-bounces at trolltech.com 
> [mailto:qt-interest-bounces at trolltech.com] On Behalf Of ?? ??
> Sent: Montag, 9. November 2009 11:41
> To: qt-interest at trolltech.com
> Subject: [Qt-interest] Using QuickTime with QGLWidget
> 
> Hi folks.
> 
> I developed a Qt application on Mac OS X Snow Leopard. My 
> main issue was drawing video frames in 3D space(using OpenGL) 
> realtime on Mac environment.
> 
> Follow lists are the summary I experienced from developing 
> the application.
> 
> Some terms may be a bit confusing:
>      Qt: the GUI toolkit
>      QuickTime 7: C interface for Apple's multimedia 
> framework including
>        functions with QT prefix.
>      QTKit: Objective-C interface for the framework.
> 
> - It is a C++ application.
> - I used the QGLWindow class for OpenGL interface.
> - I wanted to use Phonon but it seems not supporting texture 
> rendering target yet.
> - The application could have been built with only QuickTime 
> 7. But my target
>    environment was Qt on Cocoa and I felt QTKit was a bit 
> easier so I used it.
>    As a result, I had to use Objective-C partly.
> - I splited the functions related to QTKit as independent 
> framework(OS X
>    specific dynamic library package) to avoid link errors.
> - The outline of video rendering procedure is:
>      1. Retrieve an OpenGL context and format from the 
> QGLWidget and create
>         a VisualContext, which is a structure included in OS 
> X's CoreVideo
>         component, using it.
>      2. Set a callback function which notifies that a texture 
> is available using
>         QTVisualContextSetImageAvailableCallback().
>      3. Confirm a frame is really available in the callback using
>         QTVisualContextIsNewImageAvailable().
>      4. Retrieve the texture using QTVisualContextCopyImageForTime.
> - Through above procedures, we need to prevent overlapping 
> OpenGL function
>    executions then wrap up critical sections using mutex. 
> "Warning: Call
>    QuickTime OpenGL texture context functions only when you 
> are certain that no
>    other thread is making calls to the same OpenGL context 
> that the QuickTime
>    texture context is using.", Apple annotates.
> - Apple recommends to use Display Link function of Core Video 
> Component but I
>    decided not to use it. Because I felt uneasy that it could 
> corporate with Qt.
>    Instead I used QTimer to generate drawing clock.
> 
> About OpenGL context and format
> 
> - QtOpenGL's OpenGL context and format are held by 
> QGLContextPrivate which is
>    a private member(known as pimpl object) of QGLContext. I 
> retrieved the
>    object using qt_phonon_get_ptr() friend function of 
> QGLWidget, though normally
>    we should not access the object. The cx variable is a 
> context and the vi
>    variable is a format.
> - The actual data of cx and vi are depends on whether Qt 
> built with Cocoa or
>    Carbon. The former case, the widget holds NSGLContext and 
> NSGLFormat, the
>    later case the widget holds AGLContext and AGLFormat respectively.
> 
> About QuickTime
> 
> - I really need to build 64 bit application. But I gave up 
> finally for several reasons.
> - QuickTime 7 only supports 32 bit application. You must use 
> QTKit Objective-C
>    framework if you want to build 64 bit application.
> - The function of 64 bit version of QTKit is incomplete 
> subset of QuickTime 7
>    so far. For example it lacks a function directly rendering 
> video frames to
>    textures. So you have to use QuickTime 7 as well.
> - I had expected QTMovie's frameImageAttime:withAttribute:error: with
>    QTMovieFrameImageTypeCVOpenGLtextureRef passed could do 
> the task but it
>    seems only working for 32 bit.
>    http://osdir.com/ml/quicktime-api/2009-08/msg00082.html
> - IOSurface might have been usable but it is too much new 
> feature and I felt
>    anxiety about IPC is enough fast. So I didn't research it.
>    http://osdir.com/ml/quicktime-api/2009-09/msg00157.html
> - In C++ based application, AutoreleasePool is not 
> automatically allocated while
>    an autoreleased object may be allocated in QTKit call. So 
> you have to allocate
>    and release a AutoreleasePool properly, especially when 
> the console dump
>    indicating memory leaks.
> 
> I hope this post will be helpful for a person who has a similar issue.
> 
> Taiju Muto
> SGRA Corporation
> 
> 
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at trolltech.com
> http://lists.trolltech.com/mailman/listinfo/qt-interest
>