[Development] RCC XML and options

[Thiago Macieira]

On Thursday 04 September 2014 20:03:24 André Pönitz wrote:
> > That is, I’m adding logic to run rcc over a *.qrc file to produce
> > a cpp file.
> > 
> > First, I read about the -compress / -no-compress options. That is all or
> > nothing for the entire translation unit. Is there an attribute in the XML
> > (qrc) file to control this on a case-by-case basis?
> 
> No, and that's intentional. The uncompression is done once for the whole
> data set (or not at all). That's faster and more compact then compressing
> files one-by-one.

It's the difference between a .tar.gz and a .zip file: the .zip file compresses 
each contained file individually, whereas .tar.gz comrpesses the tar stream. 
Tools like RAR have the feature too: they call it a "solid package".

> > In particular, it’s counterproductive to try and compress a png file, but
> > other kinds of resources might very well benefit from compression, and
> > differing level for each.
> 
> Right. If you have such a case you can split your resources into
> two .qrc files and compress one, but not the other.

Also remember there's a trade-off: uncompressed data is loaded into read-only, 
pure and sharable portions of the memory. They can be discarded if the system 
is under memory pressure and, quite often, aren't loaded at all until needed.

If you compress, then the compressed data is read-only, pure and sharable. The 
result of the uncompression is left in read-write, impure and private memory 
segments. If the system is under memory pressure, this needs to be dumped to 
swap, instead of simply discarded.

If you have a large file and you properly use QFile::map or QResource to read 
it, you may want to keep it uncompressed.

> > (Aside: what about different compression libraries? 7zip’s lzma appears
> > to produce much smaller files than classic deflate.)

LZMA is also much slower to decompress, which can be a significant performance 
impact at runtime.

> Such features comes at a price, e.g. run time, and code complexity.
> (Runtime-wise it's actually often faster to not compress at all)
> 
> If you have a case where another compression algorithm would *really* make
> a difference you can always compress yourself using your favourite
> algorithm and then use rcc without compression.

Right. If your intention for resources is to ship files in an installer, then 
you should patiently compress to the maximum and show progress bars while you 
decompress and write to disk. That's outside of the scope of QtCore, but it 
might be something that KArchive could do for you.

QtCore and QResource are meant for fast, stream reading. We need an algorithm 
that is really fast in decompression.

> > Let me make sure I understand what that’s used for: The named function
> > needs to be called at some point to make the resources available, but is
> > otherwise not used, right? Different *.qrc files prepared by different
> > developers in different parts of the code base (in the same resulting
> > executable file) are distinguished by the ‘prefix’, right?
> > 
> > If that’s right, couldn’t the generated code contain a dummy object using
> > the “static constructor” technique to cause the registration function to
> > be called sometime before main(), and not need an exposed name at all?
> 
> That's already happening. Of course, the "static constructor" must be
> triggered _somehow_, e.g. from the dynamic loader.

The static constructor is triggered automatically by the dynamic loader. The 
big deal is to ensure that the whole .o file, including the constructor, gets 
linked into the application.

When you're doing shared libraries, that's not a problem. When you're doing a 
static library, then something must require a symbol from the .o so that the 
linker will know to include it in the output.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center