[Development] OpenGL in Qt 5.1 and onwards

[Sean Harmer]

Hi again,

On Monday 07 January 2013 08:54:41 Sletta Gunnar wrote:
> On Jan 7, 2013, at 8:53 AM, Samuel Rødal <samuel.rodal at digia.com> wrote:
> > On 12/18/2012 02:34 PM, Sean Harmer wrote:
> >> I would like to start a discussion on the future level of support for
> >> OpenGL enablers in Qt for those that are interested. So here goes...
> >> 
> >> I would like to add in a bunch more OpenGL enablers but am not sure on
> >> where to best put them. Some things I have in mind are:
> >> 
> >> 1) Classes that contain member functions for all OpenGL functions of a
> >> given version and profile. E.g. no need to use GLEW or similar. WIP
> >> version of this at
> >> 
> >> https://codereview.qt-project.org/#change,35408
> >> 
> >> This needs resubmitting against dev of course which I will do after some
> >> refactoring of the underlying code-generator and trying an experiment
> >> with a different inheritance hierarchy.
> >> 
> >> Where should this live? It fits in nicely with obtaining pointers to such
> >> objects directly from QOpenGLContext. However, it does add size to QtGui
> >> so
> >> some people may object to this being located there. The vast majority of
> >> functions are inline. The other option is to put it into a Qt3D
> >> OpenGL-enabler library and create the objects via some factory class.
> >> 
> >> At present I have a bunch of other WIP patches based upon this work but
> >> they could be refactored to resolve the necessary GL functions
> >> themselves. Using these classes would make the others easier to
> >> implement but not impossible without.
> >> 
> >> 2) An QOpenGLVertexArrayObject class. This would be just a thin wrapper
> >> around a VAO with possibly some API to make it easier to use with
> >> QOpenGLBuffer and QOpenGLShaderProgram.
> >> 
> >> This could potentially find use in QtQuick 2 so putting it in QtGui
> >> alongside the other QOpenGL* classes makes sense. I think the lack of
> >> VAO's in qtdeclarative is what makes it necessary to still use a
> >> compatibility profile with QtQuick 2. This limits the GL features we can
> >> use in conjunction with QQ2 on Mac as Apple only implement the Core
> >> Profile for GL 3.x. That is another story though.
> >> 
> >> 3) OpenGL occlusion and timing query objects. Again these are good
> >> candidates for QtGui I think but if people object they could go into
> >> Qt3D again. The timing query object in particular would be valuable for
> >> profiling the GPU time spent in rendering QtQuick 2 scenes. Iirc then
> >> right now only the CPU time is available in qtdeclarative but that is
> >> only half the story for profiling this stuff.
> >> 
> >> 4) Transform feedback object. Another small but useful candidate for
> >> inclusion in QtGui. These objects allow you to perform calculations on
> >> the GPU and write the results to a bound buffer object before the
> >> rasterisation stage. This can be used to perform animation updates for
> >> particles or physical simulations on the GPU for example. The output
> >> buffer is then used in a second pass to render with the calculated
> >> parameters. Would be very nice to integrate with QOpenGLBuffer and
> >> QOpenGLShaderProgram again.
> >> 
> >> 5) Provide support for Geometry, Tessellation Control, Tessellation
> >> Evaluation and Compute Shader support in QOpenGLShaderProgram. I don't
> >> see anything contentious with these. They are just extending
> >> QOpenGLShader and/or QOpenGLShaderProgram.
> >> 
> >> 6) Integrate support for the GL_ARB_debug_output extension into the Qt
> >> debug message system. Peppe has said he will start to look at this
> >> shortly and I think this would be an awesome feature to have available
> >> where the extension is supported.
> >> 
> >> 7) More enabler classes for textures, samplers, and higher-level
> >> abstractions. I think these would be good candidates for a Qt3D library,
> >> unless someone would really like to see a subset in QtGui.
> > 
> > I wonder if we might want to repurpose the QtOpenGL module, or introduce
> > a new QtOpenGL module that doesn't depend on QtWidgets? We could also
> > split it into an add-on and an essential part so that we can experiment
> > a bit with the APIs before they end up as essentials.
> 
> Qt3D was originally three different layers:
>  - enablers
>  - scenegraph and object loaders
>  - QML API
> 
> Right now it is all in one big lump. The ideal, if we are holding back and
> want QtGui absolutely minimal then we have:
> 
> QtGui -  OpenGL context only: you do everything yerself
> Qt3D - enablers: Contains buffers, funtion / extensions wrangling, textures,
> shader programs, etc Qt3D - scene graph and object loaders: ...
> Qt3D - QML API: ...

I agree that this would be the ideal. As you say we are already beyond this 
though since QtGui contains shader support etc.

> However, this is not what we already have in Qt Gui for OpenGL. We don't
> just provide the bare minimum. We already have 3D math classes, shader
> classes, some function resolvers, etc. If we compare to 2D painting, we
> don't just provide the bits void* of a QWindow, we provide rasterizer, font
> system, text engine, 2D vector drawing API. We make it really useful out of
> the box.
> 
> People will start looking for more OpenGL classes next to the ones they
> already found and since we put the basic enablers into QtGui, we should
> centralize them there.
> 
> This results in a few different usecaes for Qt Gui:
> 
>  - Give me an OpenGL context, I know what I'm doing: Check!
>  - I'm building an OpenGL application and need x, y and z: Check!
>  - I have this Ogre3D app I want to integrate with my app: Check!
>  - I want to embed 3D models directly into my QML apps: Use Qt3D and Check!
> 
> > That way QtGui doesn't end up growing too big with things that most
> > applications don't need. On the other hand, QOpenGLVertexArrayObject for
> > instance might make sense in QtGui down the line since I can see it
> > being potentially useful for performance improvements in scene graph for
> > instance. Though I know Gunnar has been skeptical of using these
> > wrappers in scene graph due to the additional layer of abstraction and
> > inherent performance cost :)
> 
> true, but QtGui is not only about what I personally want to use inside the
> scene graph :)

Given the current contents of QtGui and the state of Qt3D I would suggest the 
following:

QtGui continues to be the home for low-level OpenGL convenience classes. That 
is classes that have a pretty much 1:1 mapping to OpenGL objects. So things 
like:

* Shaders
* Query objects
* Texture and Sampler
* Vertex Array Objects
* Buffer objects
* Framebuffer objects

Some of these are already used by QQ2, some are not but could be in the 
future.

For higher-level abstractions I would suggest that we refactor and split Qt3D 
into a number of more fine-grained and targeted libraries. Right now it is 
quite monolithic consisting of the C++ threed library and the QML integration.

Some ideas for these:

* 3D maths (float and double versions) (or could extend QtGui 3D maths)
* OpenGL Enablers: Material; Texture unit configuration/management etc
* Geometry building/manipulation
* Encapsulation: bounding volumes, intersection testing, ray casting etc
* Animation
* Scene Management
* QML integration
* OGRE/OpenSceneGraph integrations

I was hoping to get some refactoring of Qt3D done over the vacations but then 
came down with child-induced illness, bah!

For the OpenGL version and extension classes, I think I would prefer to see 
these in QtGui as it makes sense to obtain objects of these types directly 
from the context. However, if it bloats QtGui too much then we can make use of 
a simple factory class and put them into a library inside of Qt3D.

We could redress the balance with Qt6 of course ;)

Cheers,

Sean