[Development] qmlbundle vs Qt Resource System

[Alan Alpert]

On Wed, Aug 14, 2013 at 2:10 AM, Thomas Hartmann
<Thomas.Hartmann at digia.com> wrote:
> Hi,
>
> Am 13/08/2013 18:32, schrieb Alan Alpert:
>> On Tue, Aug 13, 2013 at 8:47 AM, Thiago Macieira
>> <thiago.macieira at intel.com> wrote:
>>> On terça-feira, 13 de agosto de 2013 12:40:20, Thomas Hartmann wrote:
>>>> [...]
>>>> The only use case that came up, where .rcc should not be the solution,
>>>> is a cache for the jit. I think any cache for the jit should go into a
>>>> separate file and should not get mixed up with resource bundling.
>>>
>>> If the cache is platform-independent and future-proof, I don't see why it
>>> couldn't be added to the bundle.
>>
>> It needs to be part of the bundle, platform-independent or not. Part
>> of the qmlbundle plan was that you have a single file you can mmap in
>> to get the result of the compilation phase with maximum efficiency.
>> Part of the long-term QML plan is to JIT ahead of time, during the
>> build phase, and to even extend that to the QML parts (not just JS
>> snippets). If .rcc can't handle platform-dependent files then we'll
>> need to go with qmlbundle (or we fix QRC until it does, has anyone
>> investigated that?).
>
> I have to disagree. Mixing a cache (read write access) with resources
> (read only) in a single file/virtual file system does not make much
> sense to me.

This is not a read/write cache. This is a cache of the compiled output
from the compilation process (which might even be on the host for
cross-compiles) and could be held separately from any additional
compiled data generated at run-time.

> The cache might require a different kind of (flash) memory then the
> resources and generally mmaping the complete resource file is also not
> something you need to do.
>
> I think the cache and the resources should not be mixed up. Another
> reason is that the .rcc/rsource file is platform independent, while the
> cache is not. Imagine people copying around .rcc files for testing/hot
> fixing/patching and suddenly there is also a cache inside.
>
> Afaik .rcc are platform independent files, but I will test this properly
> until I make any real claims.
>
>>>>> [...]
>>>>> - rcc always processes a .qrc file, while qmlbundle individually add /
>>>>> remove /update files to a bundle by passing them on the command line.
>>>> There was the idea to add similar functionality to rcc. Like any command
>>>> line compression tool rcc then would take files and even (recursive)
>>>> directories and convert them to .rcc files.
>>>
>>> You're now describing a Zip file. If we're going to go anywhere there, why not
>>> use simply a Zip file?
>>
>> I think that ZIP was the original prototype of qmlbundle :P .
>>
>
> I am against using .zip.
>
> I an ideal world our bundle format would have at least these features:
>
> * zip and bzip2 compression (Huffman coding and Burrows–Wheeler
> transform). Huffman coding needs less resources while BWT yields better
> results. The user can use flags in .qrc files to control the choice. [1]
>
> * Files do not have to be compressed. Again the user can choose no
> compression in .qrc files and .rcc can check if compression is feasible. [2]
>
> * It makes sense to use one dictionary for several files to increase
> compression ratio. Since this comes with additional (memory) costs it
> should not be the default. [3]
>
> * For some use cases people might want to specify the order of files in
> the .rcc file. There are even use cases where duplicating files makes
> sense to speed up loading/seeking times. [4]
>
> * Features I do not anticipate.
>
> While .zip might support all these use cases (I do not know about 3 and
> 4) using it as a container is more work (speaking from experience), then
> just reinventing it for our specific use case. Of course using .zip has
> the advantage that 3thrd party file browsers do work. If anyone knows of
> a versatile free .zip library that supports choosing the compression
> (and not just defaults to Huffman coding) please tell me. ;)
>
> Of course we use zlib/bzlib for the actual compression/decompression.
>
> As for now I suggest we try how far we get with the current .rcc file
> format and if it is not good enough we introduce rcc2 which just has a
> different header and format, but otherwise is indistinguishable from the
> current .rcc.
> A file format with the features I mentioned above is not THAT much  work
> (speaking from experience), but I prefer doing things if really needed
> not because we can. ;)

As for now, there is no work actually planned which I am aware of in
this area. Which gives plenty of time to investigate the potential for
qrc/rcc before qmlbundle gets touched again. Given this discussion I
think we will look into the possibility of using qrc/rcc before
resuming work on qmlbundle - but that might not be for some time, and
we may be changing JS engines again or some other material change by
then ;) .

--
Alan Alpert