[Development] ICU and Windows

Koehne Kai Kai.Koehne at digia.com
Thu Mar 7 17:16:05 CET 2013

>> -----Original Message-----
>> From: development-bounces+kai.koehne=digia.com at qt-project.org
>> [mailto:development-bounces+kai.koehne=digia.com at qt-project.org] On
>> Behalf Of Joseph Crowell
>> Sent: Thursday, February 07, 2013 1:56 AM
>> To: development at qt-project.org
>> Subject: Re: [Development] ICU and Windows
>> On 02/06/2013 11:20 PM, Koehne Kai wrote:
>> >
>> > [...]
>> > That is what we should do indeed. I learned from
>> >
>> > http://userguide.icu-project.org/icudata
>> >
>> > that one can also ship the ICU data in separate .data files, located in a "ICU
>> data directory" that can be specified e.g. at compilation time. So how about
>> creating a bare minimum icudt.dll, and rather ship .data files in a well-known
>> place ($$[QT_INSTALL_ICU])? This would allow anyone with enough
>> knowledge to tailor what exactly they want to ship, while reducing the
>> footprint of "I don't care about localization for hello world" types of
>> applications.
>Alright, this is what I found out so far: You can configure ICU to either place all the data in the icudt49 library, >or in one big .dat file at a specified location, or as individual files. Having multiple .dat files is supported >too, but that requires someone deciding how they should be split up.
>Shipping the library is what we have right now, which is IMO not acceptable. Just check out the comments on the >5.0.1 release blog, there are people still caring about 20 MB overhead :) Shipping the default .dat file would mean >hat either your app doesn't have any codec support etc at all, or has everything. If we ship individual files we'd >need to ship 2345 files... that gives full flexibility, but good luck for the poor developer trying to find  out >what he needs :)
>So the bottom line for me is: _We_ have to come up with an ICU profile that contains what we consider important, and >which we want to ship in our default icudt library. If someone needs additional things he could add it by just >shipping e.g. an additional .dat file.

We discussed this a bit in the last 5.0.2 release team meeting. We could strip the data a bit, but cannot get rid of the bulk of the data without breaking stuff. So I looked into the option of storing the data separate from the .dll, in a .dat file. This has the advantage that people can (in theory, at least) customize the data without having to compile ICU on their own. Furthermore, if you know what you're doing you can also decide to _not_ ship the ICU data (though you'll get a runtime warning on startup). Anyhow, the disadvantage is that you have to deploy yet one more file with your application.

The patch I have so far is https://codereview.qt-project.org/#change,49840 . With it, you can configure an 'ICU' data dir for Qt (-icudatadir configure argument). If it is specified, QCoreApplication will try to load a .dat file from applicationDirPath()/icu, and the configured directory.

I'm not 100% happy with the patch though because we have an inherent timing problem: ICU data might be accessed & cached before QCoreApplication is created and has opened the .dat file. This happens in fact already by default, when we're trying to get the configured data path from QLibraryInfo, at which point the default codec is cached - I explicitly fix this particular issue by resetting the cache afterwards. Anyhow, it could very well be that we hit similar issues in the future.

Thoughts? Comments? Praises? ;)


Kai Koehne 

More information about the Development mailing list