[Qt5-feedback] Add internalVariant to QModelIndex

[Charley Bay]

Please forgive the long post --  I believe Stephen understands my issue, and
I want to respond with sufficient detail that clarifies as best I can.

Charley Bay wrote:
> >> See how it's done in the EntityTreeModel:
> >>
>
> https://projects.kde.org/projects/kde/kdepimlibs/repository/revisions/master/entry/akonadi/entitytreemodel_p.h#L42
> >>
> >> There is a Node type which can reference either a Collection or an Item.
> >> A pointer to a node is used for the internalPointer (see ETM::index() )
> >> <snip>,
> >>
> >> Is this what you mean to be the problem? Is my Node like the 'utility
> >> instance' you refer to?
> >
> > Yes -- I don't want the Node class at all.
>

Stephen Kelly responded:

> Ok. Now I'm starting to understand what you are saying. :)
>
> Having a Node class or something similar is quite common in QAIM
> implementations.
>
> The Qt examples use something similar, but in those examples 'the model'
> and
> 'the QAIM implementation' is one thing and implemented with the same class.
>
> http://doc.qt.nokia.com/latest/itemviews-simpletreemodel.html
>
> In this book also referenced in the docs the Node is also used for both
> 'the
> model' and 'the QAIM implementation', though it does also have a Type for
> keeping track of the type of a node.
>
>
> http://ptgmedia.pearsoncmg.com/images/0131872494/samplechapter/blanchette_ch10.pdf
>
> I mention these things only to show that they are common things to do (use
> a
> Node pointer and keep track of types with an enum). I know they are
> counter-
> examples to your case where you have a model which is self-contained and
> independent of the QAIM API (and presumably usable without it). I mention
> them so that we're clear (all of us, including the lurkers :) ) about what
> the situation is that you're trying to handle.
>

Yes -- Thank you for the examples.  I realize they represent a common
approach.  And, I concede that approach works quite well (and I use it)
when:

(1) The model only has instances of the same type; OR

(2) The model has instances of different types, but they share a common base
class (and casting-to-base is sufficient to assist in determining the
correct derived type).

Of course, at least one of those cases is typically true when the model is
created as a part of the GUI implementation (e.g., the model logically
represents application-specific code that "maps" from the business logic to
the GUI interface).

However, I'm using the term "model" in a more liberal sense:  The business
layer typically defines significant internal state, which I'm calling the
"model".  Most often, this is represented by mere "containment" with
"parent" instances owning "child" instances.  My example of "MyBook"
containing "MyChapter" instances, which contains "MyParagraph" instances is
an example.

Nearly every system has these containment models.  It is the most common
pattern in any system.  For example, the system I'm working on right now
(biomedical devices the size of a car and the price of a house) runs on
"MyProtocol" instances which contains various (different types) of
"MyInstrumentSettings" instances and numerous "MyLayout" instances composed
of "MyHistogram" instances and tons of "MyStatisticsTable" instances.  It
generates "MySample" instances which further decompose into tabular
meta-data (which could be represented as a tree) and data sets (which could
be represented as a table, but is typically filtered, and which I'd also
expose through the model).  All of these are "containment models" with no
universal IDs and no common base class.

In short, given any system, if one were to ask the question, "What state can
I expose through the GUI as a tree model?", the answer would always be,
"tons of stuff".  (I realize that most often a tree is not a good user
interface approach, I'm talking about exposing "nodes" to the GUI for the
GUI to descend down/up/laterally to explore state in a dynamic manner,
similar to how apps work on the iPhone.)

None of these "containment models" map directly to the "node" concept:  They
are all strongly typed containment, but there is no universal "ID" or "node"
type thing I can use to identify the object uniquely when starting with a
void*.  And, I assert this type of containment is *everywhere* -- all
systems have them -- and I merely want to expose that state through the GUI
through the model/view infrastructure.  These models are written,
tested/regressed, and established as third-party or reusable libraries with
no concept of GUI presentation nor coupling with Qt.

Because I violate (1) and (2) above, my only reasonable work-around is to
create an "adapter layer" which is merely a utility "mirror" (a permanently
synchronized tree) that reflects the real model, for the sole purpose of
interfacing with the GUI.  (This adapter tree is used for nothing else --
just for synchronizing with the GUI.)  This work-around results in a system
that comports with (1) and/or (2), so the work-around works.

A "summary statement" is that I'd like to merely expose my (fairly
common-and-typical) model-of-containment-of-different-types to the GUI, not
write an adapter infrastructure that I can "mount" my model into, for the
purpose of exposing to the GUI.

> For example, consider a "MyBook" object that contains "MyChapter" objects,
> > each of which contain "MyParagraph" objects.  I want to "expose" that
> > "model" to the Qt MVC by creating QModelIndex instances that reference
> > those objects, and they do not share a common base class.
> >
> > We have *many* such models, and indeed, every application has its own
> > logical "model implementations" that are not GUI and not implemented with
> > consideration of possible future coupling to the Qt MVC infrastructure.
> > These "models-of-heterogeneous-types" are terribly common.
>
> Yes, the API of 'the model' can determine how best to wrap it for the QAIM
> API. Staying with the concrete example, how do the Book, Chapter and
> Paragraph fit together? Does each element have an id()? Does Book have a
> Container<Chapter*> or Container<Chapter> elements which can be accessed
> from the book?
>

In my case, there is no "id()" or anything common among
Book/Chapter/Paragraph, and they do not share a common base class (they are
mere containment).  Yes, the Book has a Container<Chapter*> member so we can
iterate the chapters within the Book.

I realize that an "id()", or a common base class, or some other type of
adapter could help solve the problem (for example, I could maintain a
"unversal map" of void*-to-instance-and-type translations, so I can
"determine" the type identified by a QModelIndex.  However, that is merely
another adapter layer like my current mirrored-tree-of-utility-node
instances, where my node is merely (effectively):

class MyBookNode {
  public:
    void* book_or_chapter_or_paragraph_;
    enum { BOOK, CHAPTER, PARAGRAPH } enum_type_;
};

How do you map that into a internalPointer currently? If using a Node*
> solution the node might be very simple, though I know you want to avoid it.
>

My current approach/work-around is to use MyBookNode (like above), and
maintain a "tree" of them that mirror the model.  As the model changes, I
force synchronized updates to my tree-of-nodes, which is referenced through
by GUI and the model.

Since these nodes can also come-and-go (either because of user interaction
with the GUI, or because of changes on the model side, like instrument
state changes), it's kind of a lot of work.


> Copying from below:
>
> >> Our use case, very painful:  Applications commonly use "native" models
> of
> >> > heterogeneous types, and we need to create QModelIndex references to
> >> those
> >> > model "nodes".  With the current "void*", we do not know the *type* to
> >> > which
> >> > it points.  So, we create an "adapter utility" instance that *does*
> >> > know the
> >> > type.  Unfortunately, these "adapter utility" instances come-and-go
> >> > like the QModelIndex comes-and-goes,
> >>
> >> I don't think the Node* has the same lifecycle as a QModelIndex. A
> >> QModelIndex is temporary.
> >
> > For me, Node is *also* temporary.  In the example above, we keep MyBook,
> > MyChapter, and MyParagraph instances, because those are the "real" model
> > (and they do not share a common base class).
> >
> > Our work-around creates a MyBookNode class for the sole purpose of
> > exposing
> > our "real" model to the Qt QML infrastructure.  The MyBookNode instance
> > can reference any model object, and we must have a
> maintained/synchronized
> > tree
> > of MyBookNode instances that reference the *real* model.  (I'd be much
> > happier if we did not have this tree at all.)
>
> I think if your Node* is temporary you're not doing it right. But yes,
> having a MyBookNode* referring to 'the (real) model' is part of the cost
> associated with the QAIM API at the moment I think.
>

Because QModelIndex cannot have value semantics with MyBookNode (there is no
way to get it to contain or delete a MyBookNode instance), I have this
"tree" of MyBookNode instances that exist (which mirror my model).  So,
MyBookNode instances have a long life-cycle (beyond the QModelIndex
life-cycle).  However, it would be nice if I didn't need the tree (if the
MyBookNode instances could exist only as long as the QModelIndex referenced
it).

> IMHO, this issue is increasingly dramatic with QML, where interface
> > components are increasingly "independent actors", but which should
> > logically be "identifiable" through a QModelIndex-type handle to
> reference
> > the "relevant node state" from some application, especially when that
> > "relevant node state" may actually be virtual (e.g., an aspect of an
> > object or collection of objects, not an object itself).
>
> I don't think I understand this part. What does "relevant node state" mean
> to you? Something like an email object being read or unread? Is that
> something "virtual" because the read/unread state is an aspect of object
> that doesn't exist without the object? I'm trying to put things in more
> concrete terms. I think it makes these kinds of threads easier to read and
> respond to.
>

Yes -- the email read/unread context is a good example (a reference to the
object, plus a "context" bit of "read/unread").  However, I was originally
trying to permit the QModelIndex to "point to" something that may not itself
be an object.

The "relevant node state" is my attempt to assert that there can be
additional context regarding what is being referenced.  For example, the
MyParagraph can logically decompose into "lines" or "spans" or "words" or
something which is a "subset" of that paragraph.  Those may not be actual
objects, but rather, would need to be identified by a "MyParagraph*" plus
something else, like "(long index_offset_paragraph_begin, long
num_chars_in_span)".  Because QModelIndex is *designed* for exactly that (to
identify a decomposed "nested node" from a "parent node") for all the
reasons the model/view infrastructure was created, it seems like a mere
"void*" inside the QModelIndex is insufficiently extensible.

Anyway, if my description is correct, I don't see how QML is affected. You
> can already do something like myDataModel.modelIndex(model.index).isUnread
> in a QML delegate. If you have something virtual which gives you a type
> which is more complex than a bool for isUnread, I think the
> QML/Js/QMetaType/QVariant system will get in your way rather than the QAIM
> API.
>

As it relates to QML, I'm asserting that QML items are "independent actors"
that may be animated outside the confines of a layout that may be imposed,
like historically imposed from a table or a tree.  The QModelIndex is a
logical "handle" to some node within some model, and should be sufficient
for a QML component that "renders" that node to understand the "internal
type" so that component can extract the properties it needs.  In this case
too, I fear "void*" is insufficiently extensible.

In any cases like these, I would prefer a void*-and-enum, or a pointer to
some instance that could be managed with value-semantics for the same
life-cycle as the QModelIndex.


> I also don't see how any issue you hit in QML over this stuff would be
> solved by a QModelIndex::internalType() or similar.
>

My concern is that QML interfaces must handle the ability to display nodes
from multiple models, or from models with different internal types.  (I'm
aware you can get the QModel& from a QModelIndex.)  I'd like to have a
MyQmlComponent created merely from a QModelIndex.  However, because QML
components are "independent actors", without the hierarchical constraints
that may be implied when you instantiate a QTreeView (or some "heavy widget"
that references the model), the QML components need to be able to extract
all the state they need from the QModelIndex.  They won't/can't have the
extra filtering-and-model-slicing-state-and-context as might be provided by
the extra state in that "heavy widget", but rather, must be able to extract
*everything* that QML component needs from the QModelIndex (or the existing
adapter infrastructure that "starts with" a QModelIndex).  Otherwise, the
QML component *cannot* be created from a mere QModelIndex.  (That "bigger
context" class doesn't exist yet for QML if it is not a QModelIndex, I would
be interested in what the Trolls are thinking about in regards to users
creating a QML component to render a *node* within a model, not a component
for a whole *view* of the model.)

In short, QML makes components (widgets) "independent actors".  In regards
to the Qt model/view infrastructure, there is no corollary, except for
QModelIndex.  I don't want a *view* for the model, rather, I want a bunch of
QML components, each of which are independent actors, and all of which
"together" comprise some "view".  The void*-and-enum would give me enough
context to extract those node properties.  (A void* alone is insufficient.)

> Restating:
> >
> > (1)- MyBookNode only exists because QModelIndex has a typeless void*, and
> > I don't know what type it points to. <snip, cannot have
> > QModelIndex::internalId() in addition to QModelIndex::internalPointer()>
>
> Correct, now that I'm clear on what you mean there, you can only have one
> or
> the other.
>
> > (2)- The maintined/synchronized tree of MyBookNode instances only exists
> > because QModelIndex cannot perform value semantics on the MyBookNode
> > (temporary and volatile) handle.
>
> The real question is whether the Node should be temporary and why.
>

Only because "containment models" don't share a common base class (so they
cannot be exposed through QModelIndex).


> > Great happiness and joy throughout the land comes from a QModelIndex
> > design change that removes (2), or (1) (and removing (1) similarly means
> > removing (2)).
>
> I think you're saying that having the internalType() or similar would
> remove
> your need for maintaining MyBookNode*s. Just for clarity.
>

Yes.


> <snip>
>
> >> You suggest that QModelIndex [should] wholly-contain a
> MyHandleToMyObject
> >> type (through pointer that it deletes)
> >>
> >> That doesn't seem right to me for several reasons. First, it means that
> >> you'd have to new something every time index() is called and use it with
> >> createIndex() so that the QModelIndex takes ownership of it (assuming
> you
> >> could tell the QModelIndex the concrete type - it can't delete void*).
> >> QModelIndex would also have to become something like a QSharedPointer.
> >>
> >
> > I defer to the Trolls how they want to deal with that.  If value
> semantics
> > within QModelIndex were a goal, I'd be fine with a QSharedPointer-type
> > thing, with the void* being replaced with a typed QPayload* that the user
> > overrides and is always deleted by the QModelIndex, or any other
> mechanism
> > to permit a small payload for user state that is "wholly contained"
> within
> > the QModelIndex.
>
> It's not just the Trolls that would have to deal with that issue. I'd like
> to see the templates you imagine to implement this feature.
>
> I don't think the value semantics you were writing about would actually be
> usable.
>

I can think of at least three designs for value semantics, but they all
pretty much rely on turning the void* into some type (that either the
QModelIndex contains by value, or which references a type from which the
user can derive).   The advantages of these designs is that QModelIndex can
still be sizeof(void*), but yes, it would slow things down (with the
possibility of a new/delete for each QModelIndex life-cycle).

Rather than give code, I concede the (new/delete) performance hit may not be
desirable. So, I'd rather just vote for an additional "enum" inside
QModelIndex.  That makes QModelIndex "sizeof(void*) + sizeof(int)", but the
runtime performance should be the same as now.

> (I only need a void* and an enum).
> >
> > However, I would guess the easier approach would be to simply add a
> > (user-definable) enum *in addition* to the existing void*, so I can
> merely
> > cast the void* to a MyParagraph or MyChapter, as needed.
> >
> > Can I have *both* the void* and the internal id in a QModelIndex?  That
> > *would* make me happy.  (How do I create that?)
>
> So this seems to be the remaining issue. What effect would it have on the
> API? On performance? On source compatibility?
>

For the API, we would add one (overloaded) function where we could create a
QModelIndex with *both* the void* and the enum:

QModelIndex QAbstractItemModel::createIndex(int row, int column, void* ptr,
quint32 user_num);

Performance should be pretty much the same (copy construction is copying
void* and int, rather than just void*).

Source compatibility:  It's a breaking binary change, since the
sizeof(QModelIndex) is bigger (to handle the extra int).  The rest of the
interface can stay the same, though.

What is the actual problem you're trying to solve? Are you trying to solve
> an API issue because you don't think anyone should ever have to define a
> Node {}; ? Would you be ok with having the Node internalised so that the
> API
> issue is not exposed to you? Are you concerned about performance of
> maintaining the Nodes? Are you concerned about devlopment burden and
> maintenance burden for a class that uses Nodes?
>

Goal:  Containment models can be *exposed* to the model/view infrastructure
without requiring an adapter (synchronization) layer.  I assert this is a
"common" case.  IMHO, that adapter/synchronization layer is prohibitively
expensive (we don't use Qt model/view unless we absolutely must, because the
cost is so high).

For my specific concerns, it's too hard to maintain the synchronization
layer (lots of code, tedious to synchronize).  I don't have performance
concerns, except that everything I code for my adapter layer actually *slows
things down* (because of the adapter-to-model mapping and synchronization),
but I concede that performance hit usually isn't as big a concern.  It's
just too much code, and if done wrong, is easy to crash the system (in the
event the synchronization isn't perfect in all scenarios).

So, yes, development and maintenance burden.  Possibly system reliability.

It sounds like the QStandardItem API would suit you better. I know the
> QStandardItemModel API has its own problems, but let's just consider
> QStandardItem. You can create a QStandardItem and then do something like
>
> item.setData(QVariant::fromValue(myBook));
>
> and then later
>
> QVariant var = item.data();
>
> if (var.type() == BookType) ...
> else if (var.type() == ChapterType) ...
> else if (var.type() == ParagraphType) ...
>

That may be a reasonable alternative adapter layer for my
containment-models.  I see similar value-semantics-synchronization efforts,
though:  I need the QVariant to contain *both* the void*-and-enum, or a
reference to a MyBookNode that contains both.  In the end, I think it's the
same amount of code.


> If your problem is with the API, does this have the same problem for you?
> This is like internalized Nodes. Wouldn't your issue be solved by creating
> some generic wrapper which has a setData(QVariant) and data() method
> simliarly to how QStandardItem wraps a QModelIndex?
>

The API works for me -- exactly as it is.  I just need for QModelIndex to
store *both* the void*-and-enum.

For the solutions that use QVariant, including your QStandardItem API
example, I have not been able to figure out how to store void*-and-enum
within that one QVariant.  If I could figure out how to put *both* of those
into one QVariant, then I think that would give me a better work-around, but
it would still be a work-around that required significant adapter code.

All the best,
>
> Steve.
>

Thanks very much for your time on this discussion!

--charley
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.qt.nokia.com/pipermail/qt5-feedback/attachments/20110714/c9fe26bd/attachment-0001.html