[Qt-interest] Using QuickTime with QGLWidget

[武藤 大樹]

Hi folks.

I developed a Qt application on Mac OS X Snow Leopard. My main issue was
drawing video frames in 3D space(using OpenGL) realtime on Mac  
environment.

Follow lists are the summary I experienced from developing the  
application.

Some terms may be a bit confusing:
     Qt: the GUI toolkit
     QuickTime 7: C interface for Apple's multimedia framework including
       functions with QT prefix.
     QTKit: Objective-C interface for the framework.

- It is a C++ application.
- I used the QGLWindow class for OpenGL interface.
- I wanted to use Phonon but it seems not supporting texture rendering  
target yet.
- The application could have been built with only QuickTime 7. But my  
target
   environment was Qt on Cocoa and I felt QTKit was a bit easier so I  
used it.
   As a result, I had to use Objective-C partly.
- I splited the functions related to QTKit as independent framework(OS X
   specific dynamic library package) to avoid link errors.
- The outline of video rendering procedure is:
     1. Retrieve an OpenGL context and format from the QGLWidget and  
create
        a VisualContext, which is a structure included in OS X's  
CoreVideo
        component, using it.
     2. Set a callback function which notifies that a texture is  
available using
        QTVisualContextSetImageAvailableCallback().
     3. Confirm a frame is really available in the callback using
        QTVisualContextIsNewImageAvailable().
     4. Retrieve the texture using QTVisualContextCopyImageForTime.
- Through above procedures, we need to prevent overlapping OpenGL  
function
   executions then wrap up critical sections using mutex. "Warning: Call
   QuickTime OpenGL texture context functions only when you are  
certain that no
   other thread is making calls to the same OpenGL context that the  
QuickTime
   texture context is using.", Apple annotates.
- Apple recommends to use Display Link function of Core Video  
Component but I
   decided not to use it. Because I felt uneasy that it could  
corporate with Qt.
   Instead I used QTimer to generate drawing clock.

About OpenGL context and format

- QtOpenGL's OpenGL context and format are held by QGLContextPrivate  
which is
   a private member(known as pimpl object) of QGLContext. I retrieved  
the
   object using qt_phonon_get_ptr() friend function of QGLWidget,  
though normally
   we should not access the object. The cx variable is a context and  
the vi
   variable is a format.
- The actual data of cx and vi are depends on whether Qt built with  
Cocoa or
   Carbon. The former case, the widget holds NSGLContext and  
NSGLFormat, the
   later case the widget holds AGLContext and AGLFormat respectively.

About QuickTime

- I really need to build 64 bit application. But I gave up finally for  
several reasons.
- QuickTime 7 only supports 32 bit application. You must use QTKit  
Objective-C
   framework if you want to build 64 bit application.
- The function of 64 bit version of QTKit is incomplete subset of  
QuickTime 7
   so far. For example it lacks a function directly rendering video  
frames to
   textures. So you have to use QuickTime 7 as well.
- I had expected QTMovie's frameImageAttime:withAttribute:error: with
   QTMovieFrameImageTypeCVOpenGLtextureRef passed could do the task  
but it
   seems only working for 32 bit.
   http://osdir.com/ml/quicktime-api/2009-08/msg00082.html
- IOSurface might have been usable but it is too much new feature and  
I felt
   anxiety about IPC is enough fast. So I didn't research it.
   http://osdir.com/ml/quicktime-api/2009-09/msg00157.html
- In C++ based application, AutoreleasePool is not automatically  
allocated while
   an autoreleased object may be allocated in QTKit call. So you have  
to allocate
   and release a AutoreleasePool properly, especially when the console  
dump
   indicating memory leaks.

I hope this post will be helpful for a person who has a similar issue.

Taiju Muto
SGRA Corporation