[Development] templated QObjects [was: Re: We are planning to upgrade qdoc to use clang for parsing C++]

[Olivier Goffart]

Am Freitag, 26. Februar 2016, 15:56:08 CET schrieb Thiago Macieira:
> On sexta-feira, 26 de fevereiro de 2016 20:30:28 PST Milian Wolff wrote:
> > >  The main problems of templated QObject are captured more or less in
> > >  this
> > > 
> > > thread:
> > > http://lists.qt-project.org/pipermail/development/2013-March/010288.html
> > > 
> > >  Personally I still think it would be a fancy feature, a bit dangerous
> > >  to
> > > 
> > > implement maybe even dangerous to use, but really cool :-D
> > 
> > Thanks for the link. How often is the MOC layout changed in an ABI
> > incompatible way?
> 
> There's no historical pattern. There was a major break from Qt 3 to 4 and
> smaller one from 4 to 5. Qt 4 did have updates that didn't break the ABI;
> the Qt 5.0 update removed the ability to read meta objects created with Qt
> 4 moc. I don't remember how the 1→2 and 2→3 updates happened (Qt 3 still
> used register-on-load meta objects).
> 
> But since the meta object itself has a version number and the only code that
> ever reads the internal data is inside QtCore, so we could make changes
> that change the layout.  We haven't done that: all changes between major
> releases have added things without changing the layout.
> 
> That's the structure layout though. The file itself compares the
> Q_MOC_OUTPUT_REVISION macro for equality (not ordering).

Indeed, the Q_MOC_OUTPUT_REVISION could not change without breaking the 
installed moc files, if they get installed.

> > I.e. what problems would we get from having to install the
> > moc files?
> 
> Lots.
> 
> First of all, note that you're asking that
>  a) installing generated code
>  b) including such generated code from your public headers

Not necessarily installed:
 - The feature might be used only for application code or implementation 
without having the need of installing them.
 - The build system can be adapted such that moc is run on headers containing 
object template from a library, and put these generated code in the build 
directory.

> That means the headers do not compile until moc has generated its output.
> Moc is currently able to ignore missing includes and we need that when
> parsing .cpp files that contain Q_OBJECT. But doing that in headers is
> just... ick.

I don't see why this is an issue at all.
As you said we already sometimes include the moc code from the .cpp in some 
cases.

 > There's also the fact that now the generated code becomes part of your
> public ABI and will be compiled by your users. That means the code from moc
> needs to compile with their compiler settings, whichever that may be. And
> we would need to be very careful in how we change moc, because we need to
> keep users' compatibility requirements when they upgrade Qt. Their
> requirements could be stricter than Qt's (to a point).

No changes from now.
The generated code is already stricter than Qt headers themself. (we guard 
against more warnings). And since they include Qt header they already would 
have the problem with Qt headers. Also they would already have the problem 
with the moc generated files in their own application too.

> If we talk simply about non-templated QObjects, there are only drawbacks.
> The first is that the current output produces functions and data which
> would end up in *each* and *every* translation unit that included the
> output. You'd get linker errors. We could fix this by marking all the
> functions as inline, but we can't fix the data.
>
> This includes the most important data member of all: const QMetaObject *
> ClassName::staticMetaObject. It's a class-level static, so it needs to be
> defined in a single .cpp and nowhere else. Period, no fix possible.

Why would we even have to install the moc generated files for non templated 
QObject?

We can split the moc generated files in two: one with the data that is not 
installed, and one with the functions that is included by every user of the 
templated object.  =>  fix possible

> So the discussion ends here for non-template classes, at least without
> breaking Qt API.
> 
> If we were willing to break Qt API, we could remove the staticMetaObject
> class member and make it static inside a member static inline function.
> Static data inside inline functions need to be merged by the linker and the
> dynamic linker.

[This would be good step to allow unnamed class or local classes to be 
Q_OBJECT/Q_GADGET since such class cannot have static data member. But this is 
incompatible with moc anyway.  And has nothing to do with the discussion at 
hand]
This is not relevant for templated QObject so i'll skip the next paragraphes.

> [...]

 
> And then there are templates.
> 
> If we talk about template classes, then we have more problems. First, note
> the bloating and runtime-deduplication problem I mentioned above. They'd
> apply here. Though note that you can declare static data of template
> classes in headers, so we wouldn't need to break the Qt API just for this.

This bloat is a classig problem with templates in general. But that has not 
prevented them to be part of the C++ standard and used by many.

> The next problem is the meta object layout. Currently, it's optimised to
> contain static strings (an array of QByteArrayLiteral data in Qt 5, one long
> const char array in Qt 4, which I'll bring back for Qt 6). Obviously, the
> name of the instantiated class isn't constant, so the data can't be
> constant either. It stands to reason that the use of a templated class in
> the first place is the ability to use the template parameters in reflected
> members (signals, slots, invocables, properties, enums, classinfo), so the
> types and signatures for those members would need to change.

that depends how you define the names. They can as well stay constant. (not 
including the template parametter). We would just need to disable automatic 
type registration in QMetaType for pointer to templated QObject.
 
> Then we go back to whether we can keep the current Qt API: if the data isn't
> constant, can we even return const char* from the QMetaObject member
> functions like we do? If we're going to calculate the strings at runtime,
> are we even able to? What happens for:
> 
> 	signal:
> 		void somethingHappened(
> 			typename std::enable_if<std::is_default_constructible_v<T>,
> 				typename std::remove_cv<T>::type>::type);
> 
> How can QMetaObject understand what that type will be?
>
> Or we just say "the hell with it" and keep the template names in the class
> name and reflected members. Of course, you can't connect that signal above
> to SLOT(doSomething(int)) even if T was int. (no such problem with the new
> connect style)

Yes, it will be "std::enable_if<std::is_default_constructible_v<T>,
				typename std::remove_cv<T>::type>::type"
Not really usable for the string based connection syntax.  But not a problem 
for the function pointer based syntax.

> The above also points to another problem: signals need to be reflected, so
> you can't use std::enable_if like I did.

Correct.
But can be considered as a known limitation.

> Then we need to look into template members (whether in a template class or
> not).
> 
> They simply can't be extracted for reflection. So a template slot would
> never be usable in the SLOT() macro and template invocables would never be
> callable from dynamic bindings (QML, QtDBus, QtScript). And there couldn't
> be any template signals without a major overhauling of the way that signals
> are identified. See my blog "the future of moc" from 2012 for more
> information on the signal identification problem.
> 
> To work around that, we'd need an explicit list of which template members
> we'd like to be extracted for reflection in the meta object.
> 

The proposal here is about templated QObject. Not templated signals or slots.
Note that the function pointer connect allows templated slots.

> Have I convinced you? I'm only getting warmed up. I'm sure I can find more
> issues.

The issues are either non issues, or can be worked around.
There will be limitations for some corner case. But it can work in the general 
case.

> > Alternatively: couldn't moc re-create the required data from included
> > files
> > when they contain templated objects? That would solve the problem as well,
> > no?
> 
> I have no idea what you meant here.
> 
> Or, well, I do have one, but I don't think that I understood correctly what
> you're suggesting. I understood that we keep a copy of the header file's
> source in the target application and run moc at runtime to dynamically
> create the meta object, on the fly.
> 
> Since I don't think that's what you're suggesting and since the above has so
> many problems (starting with the fact that it doesn't resolve the problem),
> I'm not even going to analyse it.
> 
> Can you clarify what you meant?

It is what I re-worded at the begining of the email to avoid having to install 
generated files.  While compiling an application, the build system would re-
generate the moc generated files for the library.

-- 
Olivier 

Woboq - Qt services and support - https://woboq.com - https://code.woboq.org