[Development] qmlbundle vs Qt Resource System

[Alan Alpert]

On Tue, Aug 13, 2013 at 8:47 AM, Thiago Macieira
<thiago.macieira at intel.com> wrote:
> On terça-feira, 13 de agosto de 2013 12:40:20, Thomas Hartmann wrote:
>> My conclusion would be to deprecate qmlbundle, since it is not really
>> documented and supported (especially from the tooling side) and it does
>> not seem to have any genuine advantage, while .qrc files and .rcc files
>> have been part of Qt practically for ever and are supported by tooling.
> [cut]
>
> Sounds good.

You can't deprecate qmlbundle, as it's hasn't been initially supported
yet ;) . It's just a prototype (and yes, there's a valid question of
why it got into the release at this early point).

Once we get past the prototype stage with a real implementation,
dropping the qmlbundle code makes perfect sense. But I wouldn't advise
hastily "deprecating" it until the alternative is ready, as we may
need to resurrect it just to have something ready in time (should this
become important to someone).

>> The only use case that came up, where .rcc should not be the solution,
>> is a cache for the jit. I think any cache for the jit should go into a
>> separate file and should not get mixed up with resource bundling.
>
> If the cache is platform-independent and future-proof, I don't see why it
> couldn't be added to the bundle.

It needs to be part of the bundle, platform-independent or not. Part
of the qmlbundle plan was that you have a single file you can mmap in
to get the result of the compilation phase with maximum efficiency.
Part of the long-term QML plan is to JIT ahead of time, during the
build phase, and to even extend that to the QML parts (not just JS
snippets). If .rcc can't handle platform-dependent files then we'll
need to go with qmlbundle (or we fix QRC until it does, has anyone
investigated that?).

>> Another topic that came up was qmldir. Since Qt 5 now supports to .json
>> to describe and annotate a plugin, shouldn't we use this instead of
>> another arbitrary file format? Of course we will continue support for
>> qmldir, but we should depreciate it and use .json instead. Like for any
>> other plugin in Qt.
>>
>> The qmldir file could be replaced by qmldir.json in the pure .qml plugin
>> case.

Note that, outside Qt, the pure .qml (or mixed) case is more common I
believe than the "just a C++ plugin" case.

The other issue (possibly solvable) is that QML modules are always
defined by a qmldir file, not a plugin, which would lead to a lot of
issues (recognizing modules, finding the plugin in order to load the
metadata file) if we tried to move the functionality into the new JSON
metadata files.

> I disagree, somewhat. Please don't ask people to manually write JSON files in
> their text editors. But I can't say whether people often have to write those
> files and if they do it by hand or whether there are tools to do it.

Yes, and by hand. Every QML module needs at least one, and I don't
think that QtCreator has any support to help.

> From what
> I can tell, they're usually very simple files and are easy to generate using
> shell tools. Therefore, I'd recommend they should stay as-is.

They are indeed quite simple. The shell tools usecase is a boon, as
often you want to generate one quickly based off the contents of a
directory.

>
> This is also one of the requirements I've made for the new config API: the
> actual config files should be plain text, preferably INI, but can be cached into
> (binary) JSON.
>
>> If the plugin is converted to .rcc the .rcc file contains the
>> qmldir.json file.
>
> That sounds more like my config proposal: it's a cache, not the actual user--
> modified file.

The point of having a pre-compiled bundle should be that you have the
compiled output of the qmldir file instead. You don't need the actual
file, it's far more like the cache which Thiago suggests.

Given all those points, I'm against rewriting qmldir to be a JSON file.

>> > - qrc resources can be stored in separate files (.rcc), or embedded into
>> > the executable. Qmlbundle files are always separate files (.bundle).
>>
>> True. While most people (including me) did not know about .rcc files,
>> they are supported.
>
> I didn't either.
>
> And since I haven't done any investigation... please make sure what happens to
> the application once you load an arbitrary .rcc file. In particular, since
> resources may simply add to the root of the resource system's path, it may
> override assets that the host application needs, including those coming from
> Qt itself.
>
> Should .rcc files be considered hostile, and an application should only load a
> trusted one?

Note that it's the same with QML files. They can execute native code,
so should only be loaded when trusted.

>> > - rcc can optionally compress the content (on by default).
>>
>> If it does not already do this: rcc should not compress kwown image
>> compressed formats by default (.jpg/.png).
>
> It should compress and verify whether the compression result is worth it. If
> there's no gain, it should simply store uncompressed.
>
> One of the advantages of storing uncompressed is that QFile::map() returns the
> actual pointer to the data.

As mentioned earlier, we want the actual data ASAP. So the compression
needs to both save space and time, or it should be disabled.

>> > - rcc always processes a .qrc file, while qmlbundle individually add /
>> > remove /update files to a bundle by passing them on the command line.
>> There was the idea to add similar functionality to rcc. Like any command
>> line compression tool rcc then would take files and even (recursive)
>> directories and convert them to .rcc files.
>
> You're now describing a Zip file. If we're going to go anywhere there, why not
> use simply a Zip file?

I think that ZIP was the original prototype of qmlbundle :P .

>> > - qmlbundle has 'metalinks' that you can use e.g. store stripped /
>> > processed versions of individual files.
>> Is this used? Couldn't this be solved by file endings instead?
>
> Sounds like premature optimisation to me. A JIT cache sounds more interesting
> to me.

JIT cache is where we're going. It's because qmlbundle doesn't have
any of these interesting and useful parts yet that it's neither
supported nor documented. Nor used, it's in the repo right now either
to facilitate future development (which was unexpectedly halted) or by
mistake due to the turmoil of last year.

--
Alan Alpert