[Development] submitting a multi-dimensional container class for Qt: QNDArray

[Charley Bay]

>
> Glen Mabey wrote:

> Hello,
> >
> > For some time, I have been working on a QtCore-based class that would be
> a
> > container class of arbitrary dimensionality.  I did give a presentation
> on
> > this topic at DevDays-CA:
>

Olivier Goffart responded:

> Just my personal opinion on this:
>
> I do not think this class should be included in QtCore or even in a any
> module
> of Qt base.
>
> I think this is out of the scope of Qt. I  do not see that many uses case
> for
> normal Qt applications.
>
> Perhaps it would be a better fit in a library that does numeric calculus.
>

I attended Glen's presentation at DevDays-CA.  His introduction in this
thread is a good overview of his QNDArray class, but I'll add my impression
to focus the discussion:

Glen's QNDArray (template) class:

 (a)- approximates the "Numpy" module available in Python for
multi-dimensional data sets

 (b)- is highly performant (e.g., native C++ with attention to native
platform types)

 (c)- is designed for VERY large data sets to be stored/accessed in a
"native" format (e.g., data access is logically starting at "offsets" with
"skip-levels" between internal "values")

Olivier raises a fair point, regarding the nature of "QtCore" (what should
be included in there?), but IMHO the QNDArray makes great sense in the Qt
cross-platform-offering (some module).

Specifically:

  (1) It is an "enabling" technology that encourages Qt/C++ use where other
languages/technologies would have been used.  Glen's class is the FIRST
third-party implementation I've seen (ever) that we could actually use (we
work with data sets in the GB and TB level, and time-sensitive performance
is critical).  If added to Qt, we would consider dropping our in-house
implementations to use Glen's QNDArray (this is serious investigation at
present).

 (2) Qt/QML is now heavily focused on OpenGL.  We're looking at Glen's
QNDArray to be the "native" data sets that would be manipulated inside the
"Modern-OpenGL-static-pipeline" (great talks by Sean Harmer at DevDays-CA
on what's involved there).

IMHO, (2) is the stronger argument:  Qt needs to provide strong support for
users to write their  geometry-shaders and vertex-shaders.  This is mere
raw data-set manipulation. It seems like Qt should provide some
out-of-the-box data-set "accessor/manipulator" for those of us with very
large data sets (and of course, it should be fast).

So, I'm not picky about "which" module it ends up within, but I like the
idea of the QNDArray in some "Qt-proper" module.  The "ideal" scenario
would be if Qt's "OpenGL" or "charts" or other higher-level Qt-proper
modules interfaced directly with this "large-data-set-object". Because it
is a mere "representation-format" with no external coupling, I'd be fine
with it added to QtCore, but I'd accept other suggestions.

I agree with Olivier that the "numeric calculus" domain is somewhat
"un-bounded".  However, it seems like Qt needs to provide a data-set
*representation* format, especially since our job in Qt5+ is to implement
OpenGL shaders (which are mere raw-data-set-manipulators).

--charley
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20130109/a0b9ff54/attachment.html>