[Development] templated QObjects [was: Re: We are planning to upgrade qdoc to use clang for parsing C++]

[Thiago Macieira]

On sexta-feira, 26 de fevereiro de 2016 20:30:28 PST Milian Wolff wrote:
> >  The main problems of templated QObject are captured more or less in this
> > 
> > thread:
> > http://lists.qt-project.org/pipermail/development/2013-March/010288.html
> > 
> >  Personally I still think it would be a fancy feature, a bit dangerous to
> > 
> > implement maybe even dangerous to use, but really cool :-D
> 
> Thanks for the link. How often is the MOC layout changed in an ABI
> incompatible way?

There's no historical pattern. There was a major break from Qt 3 to 4 and 
smaller one from 4 to 5. Qt 4 did have updates that didn't break the ABI; the 
Qt 5.0 update removed the ability to read meta objects created with Qt 4 moc. 
I don't remember how the 1→2 and 2→3 updates happened (Qt 3 still used 
register-on-load meta objects).

But since the meta object itself has a version number and the only code that 
ever reads the internal data is inside QtCore, so we could make changes that 
change the layout.  We haven't done that: all changes between major releases 
have added things without changing the layout.

That's the structure layout though. The file itself compares the 
Q_MOC_OUTPUT_REVISION macro for equality (not ordering).

> I.e. what problems would we get from having to install the
> moc files?

Lots.

First of all, note that you're asking that
 a) installing generated code
 b) including such generated code from your public headers

That means the headers do not compile until moc has generated its output. Moc 
is currently able to ignore missing includes and we need that when parsing 
.cpp files that contain Q_OBJECT. But doing that in headers is just... ick.

There's also the fact that now the generated code becomes part of your public 
ABI and will be compiled by your users. That means the code from moc needs to 
compile with their compiler settings, whichever that may be. And we would need 
to be very careful in how we change moc, because we need to keep users' 
compatibility requirements when they upgrade Qt. Their requirements could be 
stricter than Qt's (to a point).

If we talk simply about non-templated QObjects, there are only drawbacks. The 
first is that the current output produces functions and data which would end up 
in *each* and *every* translation unit that included the output. You'd get 
linker errors. We could fix this by marking all the functions as inline, but we 
can't fix the data.

This includes the most important data member of all: const QMetaObject * 
ClassName::staticMetaObject. It's a class-level static, so it needs to be 
defined in a single .cpp and nowhere else. Period, no fix possible.

So the discussion ends here for non-template classes, at least without 
breaking Qt API.

If we were willing to break Qt API, we could remove the staticMetaObject class 
member and make it static inside a member static inline function. Static data 
inside inline functions need to be merged by the linker and the dynamic 
linker.

This however produces data bloat and reduces performance. For one thing, 
instead of getting the address of the meta object directly, you need to call a 
function that will return the address. For another, since we created work for 
the dynamic linker, now it needs to resolve a named relocation in places it 
didn't before. And there's still the fact that the meta object is present in 
every shared object / DLL that it got used from, even if at runtime only one 
copy is ever used.

And then there are the problems of dynamic linkers actually merging static 
data inside inline functions at runtime. This is an area where several ABIs 
fall short and that's assuming that they even try. I have one word for you 
here: dynamic_cast.

And then there are templates.

If we talk about template classes, then we have more problems. First, note the 
bloating and runtime-deduplication problem I mentioned above. They'd apply 
here. Though note that you can declare static data of template classes in 
headers, so we wouldn't need to break the Qt API just for this.

The next problem is the meta object layout. Currently, it's optimised to 
contain static strings (an array of QByteArrayLiteral data in Qt 5, one long 
const char array in Qt 4, which I'll bring back for Qt 6). Obviously, the name 
of the instantiated class isn't constant, so the data can't be constant 
either. It stands to reason that the use of a templated class in the first 
place is the ability to use the template parameters in reflected members 
(signals, slots, invocables, properties, enums, classinfo), so the types and 
signatures for those members would need to change.

Then we go back to whether we can keep the current Qt API: if the data isn't 
constant, can we even return const char* from the QMetaObject member functions 
like we do? If we're going to calculate the strings at runtime, are we even 
able to? What happens for:

	signal:
		void somethingHappened(
			typename std::enable_if<std::is_default_constructible_v<T>, 
				typename std::remove_cv<T>::type>::type);

How can QMetaObject understand what that type will be?

Or we just say "the hell with it" and keep the template names in the class 
name and reflected members. Of course, you can't connect that signal above to 
SLOT(doSomething(int)) even if T was int. (no such problem with the new 
connect style)

The above also points to another problem: signals need to be reflected, so you 
can't use std::enable_if like I did. 

Then we need to look into template members (whether in a template class or 
not). 

They simply can't be extracted for reflection. So a template slot would never 
be usable in the SLOT() macro and template invocables would never be callable 
from dynamic bindings (QML, QtDBus, QtScript). And there couldn't be any 
template signals without a major overhauling of the way that signals are 
identified. See my blog "the future of moc" from 2012 for more information on 
the signal identification problem.

To work around that, we'd need an explicit list of which template members we'd 
like to be extracted for reflection in the meta object.

Have I convinced you? I'm only getting warmed up. I'm sure I can find more 
issues.

> Alternatively: couldn't moc re-create the required data from included files
> when they contain templated objects? That would solve the problem as well,
> no?

I have no idea what you meant here.

Or, well, I do have one, but I don't think that I understood correctly what 
you're suggesting. I understood that we keep a copy of the header file's source 
in the target application and run moc at runtime to dynamically create the 
meta object, on the fly.

Since I don't think that's what you're suggesting and since the above has so 
many problems (starting with the fact that it doesn't resolve the problem), 
I'm not even going to analyse it.

Can you clarify what you meant?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center