[Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 16, 2015 07:07:56 Thiago Macieira <thiago.macieira at intel.com> wrote:

> On Thursday 15 October 2015 21:02:09 Bubke Marco wrote:
>> Actually I think Qt is not main developing library people use. It is there
>> to make the boring stuff easy, to hide the different interfaces between
>> different platforms. That is why many people use Qt,  they want to have a
>> GUI but don't want to invest to much time in it. The interesting stuff 
>> which is differentiating you from others is mostly home grown in connection
>> with much more specialized libraries. And this libraries are much more
>> important to the users. So we should support them, their interfaces and not
>> force our interfaces on them. 
>
> That's a slippery slope. If we have to support every library's interface, 
> we'll have a horrible mish-mash of an API that tries to support everything and 
> doesn't support anything really well.

I agree but do we have not that already. We have so many modules which are not up to a really big job. They are very convenient but you can run in performance trouble because they are slower. It is a trade off and we  choosed convenience. Then you try an other solution and you feel the pain of a break. You have to change so much because integrating the external solution in Qt is quite hard. What would be your feelings in that case? ;-) 

I think in that case we should make it easier to integrate the external solution. 

> I'd rather we supported the "One Qt Way" really well. Supporting other ways is 
> possible, as long as we don't cause too much maintenance trouble.

But is the "One Qt Way" not historical grown. Why did we avoid the standard library. Was it because we thought our way is better? Or because it was broken for a very long time. The usability of the standard library get better and better so we have maybe reevaluate their usefulness.

>> I know binary
>> compatibility is important for you but is it really that important outside
>> of the special linux distribution cocoon. 
>
> Yes. Lots of users don't recompile the world when Qt issues a new release, but 
> still try to upgrade.

But I  don't see how this usecase would be broken by the standard library. They have their ABI breaks like we have ours but it is not that horrible. Actually how many applications do you know which ships not their own libraries and are not part of the distribution do you know? 

> And besides, if we have to maintain binary compatibility for 99% of the API 
> anyway because of Linux distributions, then we might as well go the extra mile 
> and keep it for everyone.

Do we really have it? Is it really worth the price of all tge overhead? 

>> Is it important under Windows, 
>> is it important under Mac, is it important under embedded Linux? I think
>> the advantages are smaller than the drawbacks.
>
> I disagree.

;-) I think it is hard to argument about it because it depends so much on the context. 

> Not to mention that "embedded Linux" these days is not different from "regular 
> Linux". Even devices with 32 MB of RAM have package managers and install 
> software from a central repository.

Do you would really install a stock Qt or if it would be easy use a tailored version of Qt for your usecase. If every penny counts and the difference between a stock and an optimized version would make the difference I would choose the last. But it should be easy. 

It would nice if Qt could be compiled easily with feedback optimization and link time optimization. And if we strip automatically everything away which isn't used. :-) 

>> > That's one of the two main advantages of native code. There's no sandbox
>> > to
>> > escape from.
>> > 
>> > Qt already supports doing locale-aware comparison. We even have a class
>> > for
>> > it, so it can be done efficiently: QCollator and it supports our native
>> > string type (QString).
>> 
>> Do you like to live on a native island?
>
> Yes. I love writing native code. That's one of the biggest powers of Qt 
> compared to any other cross-platform solution out there. So, yeah, native 
> island is good.

I  meant the use of native as Qt native. I love native too but I don't love it that much that I need to many. ;-) I think the native solutions should integrate with each other quite well and not diverge to much from each other. 

> Did you mean to ask if I like constraining myself to only Qt-style APIs? The 
> answer is also yes. I hate the Standard Library API because it's confusing (to 
> me, obviously not so much to the people who wrote it), limited in convenience 
> forcing me to write more code than focusing on getting stuff done. I often feel 
> that the Standard Library tries to achieve 100% support of some tiny feature, 
> overengineering it and not focusing on making developers' lives easier. The 
> example is std::chrono.

Okay, in the last time after reevaluating the standard library I like it very much. Some concepts are still not very nice but I like many features much more since auto and lambda. And they will add parallelism to it. Then there are the coming concurrent features which looks very nice. I don't see how we can replicate them at all. And ranges are coming too which makes the STL much esier to use. 

I use chrono already and it was quite easy.

>> > Providing extra support for a character encoding that is not what QString
>> > uses falls in that 1%. Just use ICU.
>> 
>> You arguments sounds very tautological. Because it is unimportant we don't
>> have it a string class for it. It is unimportant because QString is not
>> supporting it.
>
> You're misrepresenting the argument. QString doesn't support other encodings 
> because UTF-16 is the best for the task at hand and we have too much legacy to 
> support. Because of that, QCollator only supports UTF-16.

I buy the legacy argument but I don't buy that utf 16 is the best encoding. I think because of legacy we cannot change QString but I think we should still support UTF 8 better. 

>> I know you love plationian argumentation but it would be much more effective
>> if you would try to get in the context of other and understand their
>> arguments in their context.  Showing in your own context that their
>> arguments "makes no sense" is not very useful.
>
> You're dangerously close to attacking me instead of attacking my arguments.

Okay, it was to much. I don't want it get personal. Don't see it as a discussion about personal opinions but as a way to explore a better string support for Qt.

I don't ask for an big change, we have better things to do. But ending discussion with the argument that QString is the best solution for almost all usecases is quite frustrating if you find yourself very often in a scope their QString is quite suboptimal. 

>> > That example shows how UTF-16 is better. See above on seekability of
>> > UTF-16 vs UTF-8.
>> > 
>> > The solution for this is to fix the library to accept UTF-16. When we were
>> > doing Qt 5.0, we needed PCRE to support UTF-16. Their developers were very
>> > welcoming and wrote the version that supports UTF-16, so Qt does not need
>> > to reallocate.
>> 
>> You have ever heard of Pippi Longstocking: "Widdiwiddiwitt, we make the
>> world like we wish it should be. "  or how it is translated to english. You
>> really think that you can force other larger projects to use utf16 instead
>> of utf8 if it has disadvantages for them.
>
> And why should we support UTF-8 instead of UTF-16 if it has disadvantages for 
> us?

Not instead but a little bit too. Actually I think changing QString now would be horrible for us and our users but why not add utf 8 interfaces where it matters. We have it already a little bit with QByteArray but I hope you agree we could improve that. 

> We'll have to agree to disagree with those projects. We've chosen UTF-16 and 
> we're aligned with a lot of other important API.

But would it be not better to make it easier to integrate with both?

I think we came to the point where we disagree how useful utf 8 is. If I understand you well you think utf 16 is the best solution for almost all usecases and we can ignore the rest. In my perspective QString is here and a big change is not necessary because for most usecases it doesn't matter but their are important usecases for utf 8 and they are matter. 

I hope we understand each other so far? I think the only way to decide is measurements.

>> And the PCRE is now supporting both at  runtime? Especially for large text
>> it would be very helpful if you don't need to convert them to QString
>> before you use  regular expressions on it.
>
> As far as I know, it operates entirely on UTF-16 in memory if the input was 
> UTF-16.

So how much is the difference if you use it on an utf 8 source to the native utf 8 implementation. I think we have an usecase here. If it would be much slower support for utf 8 would be useful. But how much work is it? Why not support cases which we see as useful. 

>> > Way too much code would break if we did that because we allow people
>> > access to the data pointer in QString and to iterate directly
>> > (std::{,w,u16}string don't allow that, which makes parsing them actually
>> > a lot more cumbersome).
>> I don't see the disadvantage if you have special iterators. 
>
> As long as QString contines to be backed by UTF-16, providing iterators is 
> fine. In fact, we have an iterator for UCS-4: it's called QStringIterator and 
> has been there for a year.

Good to learn something new. :-) So I think it would be quite usefull if we would have iterators for utf 8 too which would be provide UCS 4 code points. So we can exchange containers easily.

>> That is the
>> power of iterators and with the new features of C++ they get really useful.
>> But anyway,  I don't say that we have to change everything. The last time
>> we did that we broke our event system which is still not working like it
>> was before we introduced QWindow. I think we  should have an evolutionary
>> process to  adapt to the  changing environment and not try to reiterate
>> what was successful in the past.
>
> I agree with what you said in this paragraph. But it does not lead to a 
> conclusion about using UTF-8 or even providing our own UTF-8 class.
>

Introducing an utf 8 class is not that hard but providing the framework around. Collations are the problem. We have to use utf8 in QtCreator anyway so we could find out what is an useful API.  But colation support in Qt would be really helpful in the longer run.