[Development] submitting a multi-dimensional container class for Qt: QNDArray

[Glen Mabey]

On Jan 9, 2013, at 4:02 PM, Thiago Macieira wrote:

> I'm not sure it's out of scope for Qt. In fact, I don't know yet whether it 
> should be part of Qt Core or not. I guess I am the person to make that 
> particular call.

I'm very glad to hear that you are willing to consider it.

> So what I need from Glen is, at least, the proposed API docs with some 
> proposed examples of how it will work.

Back when (2 years ago?) I was trying to document this using doxygen I created examples/basic_example/basic_example.cpp and tried to include it in the class explanation, but really I recommend you look at sandbox/sandbox.cpp for some examples of how it will work.  I apologize that the docs aren't in better shape, but I will improve them.

> I need to review that to decide whether it fits Qt Core's purpose in life.
> 
>> So, I agree that too much *could* be put into QNDArray, and *that* would be
>> inappropriate for the Qt project, in my mostly-outsider opinion.
> 
> Indeed. If I look at https://codereview.qt-project.org/#change,44016, I'll 
> reject immediately. Let's start with the presence of C source files, Python 
> scripts, something called ".template" which has never been seen before, 
> examples inside the source tree instead of in the examples tree, files with 
> uppercase letters in the name.
> 
> To be accepted in QtCore, the contribution must be of a handful[*] of files in 
> src/corelib/tools, including the documentation, with examples in examples/, 
> tests in tests/auto/corelib/tools/qndarray.
> 
> [*] by "handful", I mean it literally: counted with the fingers of one hand. 
> And I do not mean binary counting (2⁵ = 32 files).

Yes, well, that is all fair and I myself am surprised at how the effort has "grown" over the years.  I'd like to believe that it hasn't sprawled, but I can't claim to be objective on that point. 

In my own defense, I'm glad that we're having this discussion here and now, and it *appeared* to me that it was the actual push (flawed though it was!) that really started this thread.

And I tried to apologize for some of these things in advance, as well as indicate that it is a Work In Progress -- and I trust that 'git mv' maneuvers are easy enough to do.  I hope that the hardest parts are done -- but an author once told me that getting the book published is harder than writing it -- so I guess we'll see how these discussions go.

I'd like to think that some of the irregularities are explainable even if they don't (yet) conform to policy.

Like the file gen_qfloat16_tables.c which produces the lookup tables used by qfloat16.  Until a couple of weeks ago that datatype was called just float16 and there were no dependencies on Qt at all (okay, there still aren't really any).  That's because I plan to post it elsewhere on the internet as a separate project if Qt doesn't want it, since I couldn't find something comparable that didn't have strings attached when I looked.  The rename to qfloat16 was just to make it more cutesy.

So, gen_qfloat16_tables.c was somehow a weird product of "create a very minimal program to create the tables" effort in the most portable way possible.  And it doesn't *ever* need to be compiled into *any* application -- only the output file, qfloat16_tables.cpp, should be (yes, it's probably missing the license headers -- sorry) but only in an application where you actually instantiate a qfloat16.

The python file autogen/autogen_files.py and the .template files are also easy targets for ridicule and I'll take it, too.  If there is a policy on "code that generates code" for Qt I missed it in the Style Guidelines …  and I couldn't seem to engage a discussion of that matter on this list, so I just pushed the issue, literally.  Sorry.  I'm still too excited about this silly container class to wait around any longer; I've got too much invested in it.

I hope that after taking a peek at the contents of the .template files you'll empathize with my approach, and I'm happy to change the file extension if that would help, and I reiterate my original offer to rewrite the python script in C++ if that will make things more palatable.  However, it doesn't answer the dilemma of where that code-generating-code should reside, so for now it remains in autogen/ .  Recommendations are welcome.

>> But I'm not proposing that the kitchen sink be put in, only the container
>> class itself, and a few very basic mathematical operations like sin, cos,
>> exp.  In fact, I probably should have down-sized the number of functions
>> that were included in the QNDAMath namespace before I pushed.
> 
> That makes sense.
> 
> We'll need to rename the class and namespace, btw. "QND" does not mean 
> anything.

Is 3D meaningful?  Surely N-dimensional is less recognizable, but it is undoubtably a widespread generalization.  The NDArray part is a direct knock-off from the Numpy nomenclature for their array class, numpy.ndarray .  Prepending with a Q and capitalizing was the best I could do to conform …  but I would love to hear any alternatives.  It does seem to be intuitive enough that boost.ndarray appeared, and I only recently became aware of this project:

http://code.google.com/p/ndarray

which also doesn't seem to find the name confusing, but I'm sure it depends a lot on the audience.

Anyway, thanks for listening --

Glen