[Development] Metatype system in Qt6

[Olivier Goffart]

On 25/01/20 17:31, Stottlemyer, Brett (B.S.) wrote:
> Apologies for reviving an old thread, but this just came up in a code review (https://codereview.qt-project.org/c/qt/qtremoteobjects/+/287828 if anyone is curious).
> 
> On 12/5/19, 11:56 AM, "Development on behalf of Olivier Goffart" <development-bounces at qt-project.org on behalf of olivier at woboq.com> wrote:
>      
>      That's a source incompatible change, and the question is if we really want it.
>      
> ...
>      
>      Do we want to automatically register the operators?
>      Do we want to automatically register the types used in signal/slots, or just
>      the properties?
>      
> I've got a bit of a different take on this question.  For QtRO (and likely 
> also for QML), I don't believe the problem of registration can be solved 
> at compile time and we will always need a good/easy way to register at 
> runtime.  I'm all for better compile time capability, but it will be problematic 
> if that limits what can be registered at runtime.

It does not reduces what can or cannot be registered at runtime.

As currently implemented (but not yet merged in dev) the following do not 
longer need registration:

  - Put a type T inside a QVariant
  - use a type as a property in Q_PROPERTY

Still requiring runtime registration:

  - Use a type as argument of signal/slot for QueuedConnection (although this 
was planed to also change)
  - Mapping from Name to QMetaType
  - QDataStream operators (*)
  - QDebug oprator (*)
  - Comparison operators (*)
  - Conversion between types.


Note that the bug experienced in qtremoteobjects/287828 seems to be unrelated 
to a change in the registration, but rather a change in the layout of QVector 
which uncovered an undefined behavior bug in the test.

> The problem I've run in to is this: how are container types of runtime types 
> are handled?  As Olivier mentioned, marshalling a custom type via QDataStream 
> first puts the type's name in the stream, then the serialization of the instance.

Since it needs the name mapping, there is no way around having a runtime 
initialization, which means that it make sense to require calling 
qRegisterMetaTypeStreamOperators<T>() before being able to serializing or 
deserializing these types.
So registering the QDataStream operators at compile time actually do not make 
so much sense..

> That means common use-cases such as 'QVector<MyType>' and 'QMap<int, MyType>' 
> are likely to be seen in the output.  What I'd like to see is container types 
> for all basic types (i.e., statically registered by Qt) registered automatically 
> (i.e., QVector<int>).  

Right now, Qt forces you to register manually 
qRegisterMetaTypeStreamOperators<QVector<Type>>() for all used type that need 
serialization/deserialisation

But QDataStream could also recognize QVector<T> and treat that specially. It 
would be able to serialize or deserialize a QVector<T> if "T" is registered, 
even if "QVector<T>" is not registered.

i.e: When QDatastream notice it doesn't know how to serialize or deserialize 
"QVector<MyType>", it does

if (type.startsWith("QVector<") {
     string_view subType = extractItemType(type); // returns "MyType";
     QMetaType subTypeMT = QMetaType::fromName(subType);
     if (auto serializeFunc = hasSerializeFunc(subTypeMT)) {
         // We need to make sure that the QSequentialIterable is always
         // registered for QVector  (this may also need changes.)
	QSequentialIterable si = qvariant_cast<QSequentialIterable>(var);
         stream << si.count();
         for (const QVariant &v : si) {
	    // Or could there be a way to avoid creating a QVariant?
             Q_ASSERT(v.metaType() == subTypeMT);
             serializeFunc(stream, v.constData());
         }
     }
}

Deserialize is the mirror of this.
It could somehow register "QVector<Foo>" dynamically. This would be a bit 
tricky since it would need to "fabricate" and destroy QVector<Foo> without 
being an actual QVector<Foo>. I have the intuition that this is feasible.

The same could be done for "QMap<A, B>".

I believe this can still be done after Qt 6.0 without even changing the 
QDataStream format, but it is going to need change on the QSequentialIterable 
front.

> In addition, we need a good way to register relevant container types when types are added at runtime.
> We are dynamically registering gadget and enum types without compile time help, 
> but that is insufficient to allow containers of those types to be used.
> 
> The problem here is different, as the type will be fully defined and there 
> won't be issues of forward declarations.  Does the template magic allow the 
> containers to be registered as well?  Or is there a way to serialize container
> types as the type itself, as well as a potentially empty list of flags, where 
> flags could include that the type is used in a container?  Otherwise handling 
> `QHash<int, MyType>`, `QHash<QString, MyType>`, etc, become problematic...

You can still register gadget and enum, and the trick above then will be used 
for QVector.


Another possibility is to always automatically register QVector<T> when 
registering T, but with something preventing to register QVector<QVector<T>>.
But then if you need QVector<QVector<T>> you are screwed.
I'm not sure it is such a good idea also because it means we are registering 
even more and that it would take more and more binary size overhead to register 
types.

-- 
Olivier