[Development] Metatype system in Qt6

[Jedrzej Nowacki]

Dnia poniedziałek, 29 października 2018 18:46:18 CET Olivier Goffart pisze:
> On 10/29/18 5:39 PM, Jedrzej Nowacki wrote:
> > Hi everyone!
> > 
> >    While main heat on the mailing list is taken by topic how to encode
> >    that we
> > 
> > are nice, friendly and respectful to each other, I would like to show some
> > side project that I had. It is a proposal for base of metatype system for
> > Qt6. You can look and comment at it here:
> > https://github.com/nierob/qmetatype. It is quite fresh and it was rather
> > a storage for functionality ideas. I haven't tried to compare performance
> > of it to the current system, but for sure it has more flexibility and I
> > believe, that conceptually it could serve us during Qt6. Anyway before
> > spending too much time on it I would like to get some early feedback and
> > questions.
> 
> The discussion in that other thread are not finished, so please wait until
> there is consensus before being nice!
Well I prefer publish it now and work in the open. I do not expect everyone 
jumping to the topic , dropping everything else, but at least some people can 
look at the code :-) There is no rush, for me it is an achieved milestone. Now 
the harder work slowly may happen.

> But thanks for caring about the QMetaType system.
Thanks for looking at the code!

> I had a short look. I think it would be usefull if you already used names
> closer to what they are supposed to be. Namespace N, P, are not so nice
> names.
N is next, P is private, kind of my personal standard. 

Action point for me:
 - updated P to QtPrivate
 - see how many name clashes I will get after removing N.

> The idea of using a single function with operation is quite a good idea I
> like it. As long as the function takes the typeid as a parameter.
> Indeed, I'm thinking about dynamic types that would come from language
> bindings: in this situation, while it is easy to allocate memory on the
> heap, it is not easy to create a new function pointer for every dynamic
> type that we would register.
Let's postpone that discussion to a moment that we agree on the extensions.

> Regarding the extension, i don't know if it is such a good idea, because you
> never know what you can rely on.
> say you have a QVariant with some type that comes from some part of your
> code, how do you know if you can print it with qDebug, or convert it to
> string, how do you register that?
> IMHO, there should not be extensions! All operation that we want to make
> available for a type should be always available. Using SFINAE to find out if
> it is available.
Extensions in that respect do not change anything, because it is about error 
handling. How to inform a user that a type is not qdebug stream-able or can 
not be converted to QString. In the current and the data driven approach you 
would need to check if the right function pointer is null or not and on top of 
this the error handling still would need to happen as the call itself could 
fail. With the proposed solution you have only one point in which you handle 
potential errors, just after the metatype call.

So let's return to QVariant concern that you expressed. I believe that if you 
plan to use a feature you need to ensure that you can. So I would propose 
QVariant constructor to look more or less like qVariantFromValue, it would do 
the "registration" it needs. It is in sync with QObject registering own 
properties types.

SFINAE would be used in extensions, as I showed in https://github.com/nierob/
qmetatype/blob/master/extensions/streams.h, still error handling is missing 
there. Actually I think it is wrong layer for detection, creating dummy 
handlers is wrong.

Action points for me:
- show in a better way that extensions can be added lazily
- improve error handling in metatype call
- highlight more common set of operations that would be used if qTypeId<T>() 
is called
- move stream SFINAE detection a bit higher in the code stack
 
> So what are the operation we want on a QMetaType:
> 
> - for QVariant, and the queued connections: copy / destruction.  Ideally in
> place. So we also need the size/alignement to be able to allocate the memory
> 
> - for qDebug, we need the QDebug operator <<
> 
> - for QDataStream, we need the operator << and >>, and an unique identifier
> that stay the same. Currently, this is the type id for builtin types, and
> the name for custom type. I suggest we use the name, if we want to keep
> compatibility with old steam, we will need to keep a mapping from old type
> id, to new name.
I'm pretty sure that you are aware that the list is far from being complete, 
so I will not neat pick on this, but how you make sure that every future use 
case would satisfied? Hiding functionality behind a function and extensible 
data struct allow _us_ to extend it if _we_ believe that costs are worth the 
gain, but it doesn't solve problem for 3rdparty code. We do see the problem 
even in our modules that needs to keep a sibling mapping from typeid to some 
functions, in particular dbus and qml(?). 

> As for the name, we can indeed find a way to extract it from parsing it from
> __PRETTY_FUNC__ as you do. It would work on every compiler we support i
> guess, but we need spcial code for that as it is not really standard.
Yes, it is compiler specific, but some version of the trick would work on all 
supported compilers, I think. Even if not we could fallback to typeid() and 
require RTTI on these potentially few weaker compilers, so my feeling is that 
it is safe to use it.

> In Qt5, we also need the name for the string-based connection syntax.
> However, I believe we can do that without relying on the name
> "as-seen-by-moc", if the moc generated code always register the types.
> (this is already almost the case)
Why almost? Is it that because of forward declared types? 

> - What about  QSequentialIterable / QAssociativeIterable? We also need
> something there.
Nothing really, currently it works through conversions. We could keep the way 
or really extract a new API that allows to do same thing. It is slightly 
orthogonal, unless I miss something.

Action point for me:
 - prototype conversions 

> In the design document you say:
>  > The whole registry is kept behind a mutex and it is very central, the
>  > mutex
>  > usage actually shows on profilers.
> 
> This used to be the case before Qt 5.7, but since then, QReadWriteMutex was
> greatly optimized, does it still show in the profiler?
Hmm true, I do not know. The list was created from the top of my head.

Action point for me:
 - remove the point

> You will need that anyway, because relying on a static variable in a
> function template does not work on MSVC as far as i know. (Last time i
> tried was a long time ago, and they had different address and value in
> different libraries) That's why we will probably need to initialize it with
> a name lookup.
Ugh, that is an ugly problem. This https://github.com/nierob/qmetatype/blob/
master/metatype_impl.h#L80 one really requires the unification of variables. 
You are right we can workaround it with a global registry and name lookup, 
but... oh.

> It is probably still a good idea to register builtin type at compile time,
> since we want to save in registering time. There are some data structures
> out there with compile time hash table that can be extended at runtime.
Yes, in addition we will need to have QMetaType::Type mapping anyway to keep 
SC.

Action point for me:
 - prototype the mapping

> So the early feedback i can give on your code is that it is a bit more
> complicated than necessary with the extension. and i think it can be
> simplified.
Because you are skipping one of the features that I would like to have. To be 
discussed ;-)

Thank you!
  Jędrek