[Development] How qAsConst and qExchange lead to qNN

[Kevin Kofler]

Marc Mutz via Development wrote:
> Sure, a trivial QWidgets program from the mid-90s may still compile and
> work in Qt 6, but as soon as said program touches Qt container classes,
> it's game over.

A lot of the changes to Qt container classes were either driven by you or at 
least praised by you, so it is really unfair for you to now blame Qt for 
those changes. Those changes were mainly driven by:
* raw performance over backwards compatibility, and sometimes even over API 
convenience, and
* interoperability with new C++ and/or STL features such as std::move,
both of which are causes that you have been heavily championing for all this 
time.

Qt could have easily kept old programs just working unmodified. It is you 
and other "modern C++" promoters that have forced it to break compatibility 
at everyone else's expense.

> Both Boost and C++ itself have a much better track record of keeping
> working code working, let alone any random C library. And this is
> actually a fair comparison, because we're comparing apples (STL
> containers) to apples (Qt containers).

The answer to that is that Qt just needs to stop doing incompatible changes. 
It is time to consider the Qt containers done and retain API and ABI 
compatibility forever. IMHO, that could and should have happened even with 
the Qt 4 version.

> Let's not kid ourselves thinking Qt is any special here. It isn't. We
> mustn't just look at one major release cycle. Qt loses projects at every
> new major release, because changes to non-core-competency parts of the
> API make porting unnecessarily complicated, so that unmaintained or
> understaffed projects[1] cannot muster the strength to be ported.

See above. Qt should not be doing major (i.e., API/ABI-incompatible) 
releases at all anymore. (Heck, you can bump the first digit if you add some 
great new feature, without necessarily breaking compatibility.)

> My goal with qNN is to make porting away _simple_. All that's required
> is to s/qNN::/std::/ and be done. No deprecation, no sanity bot, not
> even the need for local code knowledge; just simple global textual
> replacement. And no regressions, if you first switch to C++NN, and only
> then do the replacement. We can even retain the qNN symbols until Qt
> requires C++(NN+3), because the qNN headers will be nothing but a long
> list of using std::foo; at that time.

How about #define q17 std? Or even #define qAsConst std::as_const? If it is 
really a drop-in replacement as intended, that should work. (And in the 
unlikely event of a name conflict with user code, #undef can be used as a 
quick workaround.)

> It's not really acceptable that such trivial ports should be subjected
> to all the same (or, apparently since it's done in bulk, more
> restrictive) requirements than for deprecation of core-competency APIs.
> The more so as I must have missed the outcry of developers when we
> inflicted
> 
>    // Universe A
>    Qt 1      Qt 2     Qt 3                     Qt 4+5             Qt 6
>    QGList -> QList -> QPtrList / QValueList -> QList / QVector -> QList
>                                                QLinkedList

Oh, I did complain about the QList changes in Qt 6. You, on the other hand, 
have proposed (on 2017-03-18) to just "Kill QList in Qt 6" which would have 
broken existing application code even more than the change that was 
implemented.

And I was not yet on this list when the Qt 4 changes happened, or I would 
have complained about those, too.

> on an audience that could, instead, have had
> 
>    // Universe B
>    Qt 1      Qt 2-4         Qt 4-5         Qt 6           Qt 6-7
>    QGList -> std::vector -> std::vector -> std::vector -> std::vector
>    Cfront    C++98          C++11/14       C++17          C++20/23

But std::vector is much worse than any of the Qt containers above: no 
implicit sharing, uglier and less complete API, no amortized O(1) prepending 
optimization.

The Qt 5 QList also had efficient random insertions/removals for large 
structures, a feature that was actually LOST in the Qt 6 version. (Yes, you 
can explicitly store pointers in the list to simulate the old behavior, but 
that means less convenient API.) But std::vector never had that feature to 
begin with.

> To hold users, a project must maintain _long-term_ API stability, not
> rewrite the container classes for every major release.

Indeed. So why have you not supported keeping the Qt 5 QList in Qt 6 then?

> So, sorry, but as a user of Qt I'd really like to use those stable STL
> classes, if only your volatile APIs let me. I'd rather I had done that
> _one_ port between Qt 1 and 2 than all those ports in Qt 1-6.

I would rather have kept the Qt containers without constant porting and 
without losing features (which the STL never had to begin with).

Did you know that Qt 3 containers actually had defined behavior on 
overflows? (An out-of-bounds read would always return 0, which was 
especially useful for strings.) That was dropped in Qt 4 in the name of 
performance, and so the Qt 4 ports of applications that had relied on this 
partially documented Qt 3 behavior suddenly started to crash and have 
security issues. (For QString, the value was explicitly documented as being 
QChar::null. For QValueList, it was documented that "an undefined value" is 
returned, which means it should at least not segfault or assert. In Qt 4, 
these sentences were silently dropped from the documentation, and the code 
changed to either assert or just let the heap overflow happen, depending on 
debugging settings.) This was not even documented in the "Porting to Qt 4" 
document: back then, I noticed the discrepancy only when my code crashed 
after porting.

> Anyway; to all those who disagree when I say Qt should concentrate on
> its core competencies and stop meddling with container classes, shared
> pointers, etc, I say this: which of the two universes above would you
> rather have lived in these past 30 years? A or B? Be truthful!

In neither, see above.

> Because, you see, whenever we phase out some Qt NIH API, we beam a small
> part of your (and our!) code from Universe A into the Universe B. I
> _really_ can't understand why anyone would complain. Must be a case of
> recency bias or something like that...

Because the STL containers are objectively worse. They may be faster if you 
use them correctly (and much slower otherwise, because, e.g., you may be 
copying the whole vector including all objects it contains instead of 
incrementing a reference count), but they are worse in all other aspects: 
the function names are terse and confusing (e.g., empty) and use bizarre 
terminology (e.g., front and back instead of first and last), they do not 
have implicit sharing, they do not have methods for O(n) operations (e.g., 
contains) when you need them (forcing you to use some freestanding function 
from the really ugly <algorithm> API or to write the loop by hand), etc.

C++ would be a much better language without the STL (or with a different 
STL, roughly the Qt 4 containers and algorithms, maybe with some of the Qt 3 
features such as the safe zero value for out-of-bounds reads). I miss the 
times where STL support in Qt was optional and Qt applications were able to 
completely ignore the STL's existence.

See, e.g., the priorityqueue.h header in:
https://www.tigen.org/kevin.kofler/fmathl/dyngenpar/dyngenpar-12.tar.xz
(GPL version 2 or later) for how I had to sanitize an STL container to be 
actually usable. (Before I added the wrapper, I was not even able to figure 
out how to construct an std::priority_queue without needlessly copying the 
constructed temporary, deep-copying the entire queue! Mind you that this was 
C++98 code, so std::move was not a thing. The implicitly-shared wrapper 
class fixed that, and also allowed me to fix the ugly API.)

> Speaking of the devil, from my pov, qAsConst isn't "small enough" to
> survive. Fighting off patches that try to make the deleted rvalue
> overload "do something" is _also_ maintenance work, even if it doesn't
> result in a code change.

Well, the thing is that if you were not so set on wanting to eventually 
remove qAsConst, you would not have to fight off those patches. So it is 
kind of a circular argument: you want to remove qAsConst because you do not 
want somebody to make it better than the STL version, and you do not want 
somebody to make it better than the STL version because you want to remove 
it.

qAsConst could be a lot more useful without that constraint of being bug-
compatible with the STL version. In particular, it could make ranged for a 
drop-in replacement for Q_FOREACH without having to introduce explicit 
intermediate variables for the temporaries. Though then again, at that 
point, this:

> Finally, I'll bet my behind (pardon my French) that at some point in the
> not-too-distant future someone will appear on Gerrit with a patch to
> change Q_FOREACH to use C++20 ranged-for-with-initializer:
> 
>    #define Q_FOREACH(decl, expr) \
>       for (const auto _q_foreach_c = expr; decl : _q_foreach_c)

sounds like the better approach. Any caveats with that idea? In other words, 
what will it break?

> And I'd probably approve it, because then that thing can actually just
> be replaced with the expansion everywhere, and then, finally, be deleted.

But then I do not understand what we have to gain from removing this simple 
one-line macro that (hopefully) keeps old code compiling with no porting. 
Does that not contradict your first paragraphs?

        Kevin Kofler