[Development] How qAsConst and qExchange lead to qNN

Marc Mutz marc.mutz at qt.io
Fri Nov 4 16:00:50 CET 2022


Hi,

After getting my head washed by Volker, lemme provide background on 
these two functions.

TL;DR: we created real maintenance and porting problems by not removing 
stop-gap functionality in a timely fashion, qNN presented as a way to 
ensure this won't happen again.

Both qAsConst and qExchange are 1:1 reimplementations of std 
functionality we couldn't rely on back then (as_const is C++17 and 
exchange is C++14), so they're exactly equivalent to the std versions. 
Or were, when they were added.

Neither std::as_const nor qAsConst have changed, so the replacement is 
easy: s/qAsConst/std::as_const/. It cannot not compile because qAsConst 
comes from qglobal.h and that transitively includes <utility>. This is 
what was intended from the get-go:

https://codereview.qt-project.org/c/qt/qtbase/+/114548
 > Intended as internal API, but not put into the QtPrivate
 > namespace to make it simpler to use.
 >
 > We could wait for std::as_const, but that is far, far
 > away (just entered the current C++17 draft as of this
 > writing), and the Qt containers with their tendency to
 > detach are a problem _now_.

I don't remember why sometime later we (I) documented it as public API, 
but I failed to add the magic sentence

 > Use std::as_const if you can (C++17). qAsConst will be removed is a
 > future version of Qt

We seem to be removing these kinds of sentences from API (currently on 
Gerrit for Q_FOREACH), but I don't think we should.

Anyway, that's qAsConst. It does what std::as_const does, so moving to 
The Real McCoy is trivial.


Unfortunately, as it always seems to happen, both std::exchange and 
qExchange changed. They're still 100% equivalent, but qExchange is now 
equivalent to std::exchange in C++20 (constexpr) and C++23 
(conditionally noexcept, technically a DR), so we can _not_ blindly 
replace qExchange with std::exchange, because we only require C++17.

It is important to dwell on the difference between the two:

std::as_const never changed, so qAsConst never had to.
std::exchange changed, so it was felt that qExchange had to, too.

The outcome is that we don't need to maintain qAsConst, and can 
trivially port away from it, but we have (continued) to maintain 
qExchange, and can _not_ trivially port away from it.

What was the mistake? The mistake was to not limit the qExchange 
implementation to the C++14 version, but directly going for C++20 
semantics (constexpr; mea culpa) and therefore not being able to remove 
qExchange when we _could_ (in 6.0, when a C++14-only qExchange and 
std::exchange would have been semantically identical). Instead, we kept 
dragging it on with unclear semantics (C++14 or C++20?), until C++23 
semantics were added to qExchange() extending the lifetime even further. 
Had we limited qExchange to C++14 and removed qExchange when we had the 
chance, this lock-in would not have happened.


But why do the deprecation _now_?

It turns out, when we decomposed qglobal.h to speed up compilation (a 
high-prio request from users), we ended up with a clean qglobal.h 
including only other includes, except qAsConst and qExchange for which 
we didn't find a good home. My idea, then, was to deprecate qAsConst and 
qExchange, introducing q20::exchange (see below), so that they could 
stay in qglobal.h, itself to be deprecated in favour of the more 
fine-grained headers and port the code base away from qAsConst and 
qExchange to pave the way for rolling out the finer-grained headers. It 
turns out I was too slow and qAsConst and qExchange are now in 
qttypetraits.h (go figure), and we have problems deprecating them 
_there_, because the Qt deprecation macros aren't defined, yet, when 
that file is included in some TUs.

One problem after the other. This is just to illustrate that there's a 
very real cost associated with keeping stop-gap reimplementations of std 
functionality around for longer than absolutely necessary.



So, late in Qt 5 I came to experiment with how we could break this 
vicious cycle: On one hand, we want to provide some std library 
reimplementations when we just don't want to or cannot wait for them to 
become available. But OTOH we want these to be stop-gaps, until we can 
depend on a version of C++ in which they ship, and remove them as soon 
as possible to avoid the above-mentioned trap where we need to maintain 
them indefinitely. And certainly, we don't want to follow the std 
implementation changes lest we never catch up with the std version.

For some time, I thought just making these functions private would be 
enough (qmemory_p.h for q_make_unique), but private API means we can't 
use it in public headers. Also, there's no indication when we can remove 
the stop-gap again. Was make_unique C++14 or C++17? Hmm, hmm...

Finally, there's the problem of whitespace: assume we've solved the 
problem with use in public headers, and I could write

    auto p = q_make_unique<Widget>(foo, ~~~~,
                                   bar, ~~~~);

I can safely s/qt::make_unique/std::make_unique/, but indentation is now 
messed up:

    auto p = std::make_unique<Widget>(foo, ~~~~,
                                   bar, ~~~~);

Enter the qNN mechanism.

For C++17 void_t, I therefore decided to place my reimplementation into 
the q17 namespace:

   namespace q17 {
      // like std::void_t
      template <typename...>
      using void_t = void;
   }

That didn't end up being merged, but I've used this pattern in several 
projects afterwards, even adding it to KDToolBox.

The idea is that the namespace both hints at the C++ standard used, as 
well as having the same length as the std namespace. Now, if I replace 
q17::void_t with std::void_t, indentation etc will be maintained.

It gets better: I don't even need to know what exactly we have 
reimplemented of the C++17 std lib. I can just globally search and 
replace q17:: with std:: and  <q17xxxx.h> with <xxxx>.

Naturally, libc++ will be lacking some features that other STLs have 
long since implemented (*cough* pmr, *cough* filesystem), but these tend 
to be more complex than what we'd reimplement in qNN, anyway, so just 
replacing qNN:: with std:: when re require C++NN is still going to work.

To minimize risk, we implement the qNN functionality with the following 
guarantee: If an expression using qNN compiles and works, then the same 
expression with std:: instead of qNN:: will compile and work the same 
way. IOW: we can leave functionality out vis a vis std, but what 
implement must be 100% compatible with the std version: no Qt-ish names, 
no funny optimisations std doesn't have, etc.

And if std::foo changed in C++MM, MM > NN, and I've already implemented 
qNN::foo, I'm not going to change the implementation in qNN, I create a 
new implementation in qMM.

Conversely, as a user of these functions, when faced with implementation 
of q14::exchange and q20::exchange, you should pick the one that has all 
the features you need, not blindly pick the latest one. So far, the 
higher-version version has a small comment that says what's changed:

    // like C++20 is_sorted (ie. constexpr)

Finally, in order to check that there will be no problems when we make 
the switch to a newer C++ version, each qNN function or type will 
automatically be an alias to the std version, if the latter is available:

   // like C++20 is_sorted (ie. constexpr)
   namespace q20 {
   #ifdef __cpp_lib_constexpr_algorithms
       using std::is_sorted;
   #else
       ~~~ our implementation ~~~
   #endif
   }

So when you make the switch from q20 to std, it's literally no change at 
all.

On the downside, this means we can't use these types in public API, 
which is why QSpan cannot be q20::span. Mostly, there's stuff useful for 
implementation, though, so that's less of a restriction than it might seem.

There are quite a few components in namespace q20 already, and we even 
have one that's not in the standard, yet: qxp::function_ref (xp for 
eXPerimental).

All qNN headers contain this warning:

   //
   //  W A R N I N G
   //  -------------
   //
   // This file is not part of the Qt API. Types and functions defined
   // in this file will behave exactly as their std counterparts. You
   // may use these definitions in your own code, but be aware that we
   // will remove them once Qt depends on the C++ version that supports
   // them in namespace std. There will be NO deprecation warning, the
   // definitions will JUST go away.
   //
   // If you can't agree to these terms, don't use these definitions!
   //
   // We mean it.
   //

Thanks for reading this far,
Marc


More information about the Development mailing list