[Development] QtCS - ICU and Localization session

[John Layt]

On Friday 26 Jul 2013 12:23:43 Thiago Macieira wrote:
> On sexta-feira, 26 de julho de 2013 19:33:40, John Layt wrote:
> > I've started work on refactoring QLocale and my existing ICU api code
> > to try make this work and should have a working proof-of-concept in a
> > couple of weeks.  At the very least, even without making the new
> > classes public this should result in a cleaner QLocale, an ICU backend
> > for use on Linux and QNX, improved Mac and Win32 support and smaller
> > library size on most platforms.
> > 
> > Thoughts?
> 
> Thanks for the braindump and analysis, John. It's appreciated.
> 
> I find myself agreeing with almost everything you said. The one thing I
> disagree with is maintaining our old database in Qt. Past experience teaches
> us that anything but the default build will bitrot and will be hard to
> maintain. It hasn't been one year that we started using ICU instead of
> iconv and the iconv code already has shown build issues every now and then
> -- and no one knows whether iconv on Windows is supposed to work.
>
> So I would request that we seriously consider deprecating and later dropping
> completely our own database. I would put the implementation on two levels:

My main reason for keeping the old database and code about was as a fallback 
for any platform where the host system proved inadequate or we didn't have 
time to implement it, i.e. Win32, Android and any other POSIX or embedded 
platform.  I don't see it as a long term support option, I want to be rid of 
the overhead.

> 1) baseline and mandatory: system/user config
> 	backends: POSIX, Win32, OS X, BB services, whatever Android uses
> 	(the first four are already present, the POSIX one needs work)
>  2) database support (optional)
> 	backends: ICU, OS X's ICU wrapper, Win32, ICU4J via JNI

Both Android and QNX/BB are POSIX in the absence of ICU, and the Mac ICU 
wrapper is their main api unless you use POSIX.  I really would prefer to use 
ICU as the backend on both Linux and QNX given the serious shortcomings in 
POSIX (no calendar systems or proper currency support, etc) and the knowledge 
they are always installed, and I think the BB team are keen for us to use ICU 
as well.

> The first level is the implementation that gives the application access to
> the user's current locale settings. For the overwhelming majority of
> applications, that's actually quite acceptable: users don't often run
> applications in different languages than their OS, or parts of the
> application.
> 
> In turn, it takes the burden off the backends for the second level. If the
> ICU4J backend (for example) is missing or slow, it won't be a big deal.
> 
> That would leave us with the task of implementing a very good "system/user
> config" module. That would also mean bringing the iconv converter back to
> working condition.
> 
> The one problem I see is QCollator. I don't see the baseline implementation
> providing enough data for us to implement said class. I'll be happy to be
> proven wrong, though. But if I'm right, my suggestion is to remove this
> class from QtCore -- it's still private and not used anywhere. We can
> provide it elsewhere and make the ICU dependency explicit there.

I'm not sure about creating a separate module for just QCollator, 
QNumberFormatter and QDateTimeFormatter, it seems overkill for only 3 classes, 
but it has the advantage of that explicit ICU dependency.  One problem could 
then be how to plug the ICU backend in for QTimeZone and QCalendarSystem which 
must live in QtCore so they can be used with QDateTime.  Also, if we had an 
ICU backend in QtCore for QLocale then we'd also be duplicating code between 
QtCore and QtICU as you could't assume QtCore was built with ICU support.  I 
think simple practicalities rule out a separate module

For 5.2 let's aim for an interim target:
1) Add QTimeZone using the host systems with ICU as an optional choice
2) Refactor QLocalePrivate to use only the host systems on Win32 and Mac and 
an ICU backend for Linux and QNX, and use the old Qt backend for everything 
else.
3) Add QCalendarSystem using the host systems / ICU
4) Look at a better locale-aware sort but not yet the full QCollator

For 5.3 we can then look at adding an advanced api and other backends, and 
hopefully remove the Qt database/backend.

John.