[Development] Metadata for loading QML plugins (was: qmlbundle vs Qt Resource System)

[Alan Alpert]

On Thu, Aug 15, 2013 at 11:23 AM, Thiago Macieira
<thiago.macieira at intel.com> wrote:
> On quarta-feira, 14 de agosto de 2013 12:11:17, Alan Alpert wrote:
>> Basically what the engine needs is to find the types loaded into that
>> module URI/version. So either the plugins in that abstract cloud need
>> to be all loaded (in the QML engine sense, primarily calling
>> registerTypes) or the URI/version needs to be able to specify a plugin
>> to load. So in that abstract cloud case we'd query the cloud to see if
>> it has a plugin matching that URI/version, and load that one if we
>> find it.
>
> I'm not sure I followed you.
>
> Is this the processing of the import statement? Or is this the instantiation
> of an object?
>
> For example, if I have:
>
> import Foo.Bar 1.0;
> Foo {}
>
> At what point do you want to run the plugin's code? I imagine it's during the
> import, but I could be wrong and the engine might not actually run anything
> until the type is instantiated.

At the import statement we run the plugin's code (if not loaded). By
the time you get to instantiating an object we've already built the
internal data structures that count as an 'abstract cloud of modules',
and we just use those. The import statements then load those types
into a certain namespace for use in that file's scope, but that's from
the internal data structures now.

> We might be trying to solve two distinct problems here and not talking about
> the same thing...

I didn't know that there is a problem we're trying to solve here. It
started with speculation about far-future direction and I thought we
were still there (just digressing into understanding the current
structure first).

>> For additional context, here's the current* process for module imports
>> (e.g. import QtQuick 2.0)
>>
>> - Does the import match the name/version of a registered, protected,
>> C++ module? If so load the types registered in that module.
>
> You've lost me here. I have no clue what a protected module is, I don't know
> how you'd register one, and I don't know what you mean by "load the types".

The import statement loads types of a module into the engine. This
becomes the data structure which the rest of the file can use to
instantiate types with.

The protected stuff is a bit confusing, because it's one place where
we had to layer something on which should have been part of the
original design, but in a backwards compatible way.
EXPLANATORY DIGRESSION:
Originally there was no protected concept. We realized that we needed
one when some set of components (I forget which exactly) started to
register new types into the QtQuick namespace. We can't break that
functionality in the generic case, but we added a "module version"
statement to qmldir files which would protect them from having other
types loaded into that namespace. As the C++ registration equivalent
is coming in, it needs to be checked first so that it can't be
overwritten should a module be found. This also fixes a performance
issue, as the engine currently must query the file system even in
cases where all known types have been registered already. If QML could
be rewritten from scratch (it's too early/late for that, depending on
your perspective), protection for modules would probably be automatic.
I have not yet seen a legitimate usecase for injecting types into a
module from another source.

>> - Can the module be located in local import paths?
>
> I imagine this is the "import Foo.Bar 1.0" statement above. The engine sees it
> and tries to determine if there's anything that can provide Foo.Bar 1.0.
> Correct?

Yes, which usually involves querying the file system for a module in
an import path.

>> --Search for each import path for
>> $PATH/Name/with/dots/converted/to/slashes/qmldir then
>> $PATH/Name/with/dots/converted/to/slashes.MajorVersion/qmldir
>> --If the qmldir is found, load the module (load the plugin, register
>> types in the plugin, register types defined in the qmldir file)
>
> Ok, this is one mechanism today.
>
> We're wondering if we can add a new mechanism. The plugin system can provide
> you with the contents of the qmldir file for you. You don't have to scan for
> it.

It's possible to add a new mechanism if there's a third,
non-conflicting, way to find the plugin - but this then skips over the
whole qmldir file contents. The qmldir is there as part of how a
module URI can be found from the filesystem. Once we have the location
of the plugin there is no need for the qmldir file contents (in the
plugin-only case, and the other cases can't be solved by the plugin
system anyways ;) ).

Note that if you have the plugin loaded manually you can also call
registerTypes manually with what's in there today.

> The most basic query would be "give me all qmldir" and you'd find the one you
> want. A more elegant way is to store some key information outside the qmldir
> file, like the module name and version ("Foo.Bar" and 1.0).

So if I'm following you correctly, you're talking about an approach
where on startup all available plugins have their metadata loaded as a
database of URI->pluginpath pairs. Then an import statement could
query this database instead of the filesystem in order to find the
plugin location for that URI/version.

I can see that being more efficient for statically linked plugins (not
actually an issue I'm looking into), so having that as an extra path
might help if that's what you're trying to solve. For the other cases,
it seems like just trading an existing 'database' (the file system)
for an internal one.

Note that when "scanning" for the qmldir files on the filesystem,
we're basically just querying the exact same two keys we'd query in
that database (except with /qmldir appended).

>> - Does the import match the name/version of a registered, C++ module?
>> If so load the types registered in that module.
>
> Lost me here again.

You can register types outside of plugins. For example:
qmlRegisterType<QTimer>("App", 0, 1, "MyTimer"); in your main.cpp will
register a type into an App 0.1 module. Then an "import App 0.1" line
will find this module in the engine data structures, without touching
the file system (after that optimization is in...) or any qmldir
files.

>> - Can the module be located in the remote import paths?
>
> Remote paths?

QML2_IMPORT_PATH="http://www.myweb.com/imports" I believe is valid.

--
Alan Alpert