[Development] How qAsConst and qExchange lead to qNN
Marc Mutz
marc.mutz at qt.io
Fri Nov 4 16:00:50 CET 2022
Hi,
After getting my head washed by Volker, lemme provide background on
these two functions.
TL;DR: we created real maintenance and porting problems by not removing
stop-gap functionality in a timely fashion, qNN presented as a way to
ensure this won't happen again.
Both qAsConst and qExchange are 1:1 reimplementations of std
functionality we couldn't rely on back then (as_const is C++17 and
exchange is C++14), so they're exactly equivalent to the std versions.
Or were, when they were added.
Neither std::as_const nor qAsConst have changed, so the replacement is
easy: s/qAsConst/std::as_const/. It cannot not compile because qAsConst
comes from qglobal.h and that transitively includes <utility>. This is
what was intended from the get-go:
https://codereview.qt-project.org/c/qt/qtbase/+/114548
> Intended as internal API, but not put into the QtPrivate
> namespace to make it simpler to use.
>
> We could wait for std::as_const, but that is far, far
> away (just entered the current C++17 draft as of this
> writing), and the Qt containers with their tendency to
> detach are a problem _now_.
I don't remember why sometime later we (I) documented it as public API,
but I failed to add the magic sentence
> Use std::as_const if you can (C++17). qAsConst will be removed is a
> future version of Qt
We seem to be removing these kinds of sentences from API (currently on
Gerrit for Q_FOREACH), but I don't think we should.
Anyway, that's qAsConst. It does what std::as_const does, so moving to
The Real McCoy is trivial.
Unfortunately, as it always seems to happen, both std::exchange and
qExchange changed. They're still 100% equivalent, but qExchange is now
equivalent to std::exchange in C++20 (constexpr) and C++23
(conditionally noexcept, technically a DR), so we can _not_ blindly
replace qExchange with std::exchange, because we only require C++17.
It is important to dwell on the difference between the two:
std::as_const never changed, so qAsConst never had to.
std::exchange changed, so it was felt that qExchange had to, too.
The outcome is that we don't need to maintain qAsConst, and can
trivially port away from it, but we have (continued) to maintain
qExchange, and can _not_ trivially port away from it.
What was the mistake? The mistake was to not limit the qExchange
implementation to the C++14 version, but directly going for C++20
semantics (constexpr; mea culpa) and therefore not being able to remove
qExchange when we _could_ (in 6.0, when a C++14-only qExchange and
std::exchange would have been semantically identical). Instead, we kept
dragging it on with unclear semantics (C++14 or C++20?), until C++23
semantics were added to qExchange() extending the lifetime even further.
Had we limited qExchange to C++14 and removed qExchange when we had the
chance, this lock-in would not have happened.
But why do the deprecation _now_?
It turns out, when we decomposed qglobal.h to speed up compilation (a
high-prio request from users), we ended up with a clean qglobal.h
including only other includes, except qAsConst and qExchange for which
we didn't find a good home. My idea, then, was to deprecate qAsConst and
qExchange, introducing q20::exchange (see below), so that they could
stay in qglobal.h, itself to be deprecated in favour of the more
fine-grained headers and port the code base away from qAsConst and
qExchange to pave the way for rolling out the finer-grained headers. It
turns out I was too slow and qAsConst and qExchange are now in
qttypetraits.h (go figure), and we have problems deprecating them
_there_, because the Qt deprecation macros aren't defined, yet, when
that file is included in some TUs.
One problem after the other. This is just to illustrate that there's a
very real cost associated with keeping stop-gap reimplementations of std
functionality around for longer than absolutely necessary.
So, late in Qt 5 I came to experiment with how we could break this
vicious cycle: On one hand, we want to provide some std library
reimplementations when we just don't want to or cannot wait for them to
become available. But OTOH we want these to be stop-gaps, until we can
depend on a version of C++ in which they ship, and remove them as soon
as possible to avoid the above-mentioned trap where we need to maintain
them indefinitely. And certainly, we don't want to follow the std
implementation changes lest we never catch up with the std version.
For some time, I thought just making these functions private would be
enough (qmemory_p.h for q_make_unique), but private API means we can't
use it in public headers. Also, there's no indication when we can remove
the stop-gap again. Was make_unique C++14 or C++17? Hmm, hmm...
Finally, there's the problem of whitespace: assume we've solved the
problem with use in public headers, and I could write
auto p = q_make_unique<Widget>(foo, ~~~~,
bar, ~~~~);
I can safely s/qt::make_unique/std::make_unique/, but indentation is now
messed up:
auto p = std::make_unique<Widget>(foo, ~~~~,
bar, ~~~~);
Enter the qNN mechanism.
For C++17 void_t, I therefore decided to place my reimplementation into
the q17 namespace:
namespace q17 {
// like std::void_t
template <typename...>
using void_t = void;
}
That didn't end up being merged, but I've used this pattern in several
projects afterwards, even adding it to KDToolBox.
The idea is that the namespace both hints at the C++ standard used, as
well as having the same length as the std namespace. Now, if I replace
q17::void_t with std::void_t, indentation etc will be maintained.
It gets better: I don't even need to know what exactly we have
reimplemented of the C++17 std lib. I can just globally search and
replace q17:: with std:: and <q17xxxx.h> with <xxxx>.
Naturally, libc++ will be lacking some features that other STLs have
long since implemented (*cough* pmr, *cough* filesystem), but these tend
to be more complex than what we'd reimplement in qNN, anyway, so just
replacing qNN:: with std:: when re require C++NN is still going to work.
To minimize risk, we implement the qNN functionality with the following
guarantee: If an expression using qNN compiles and works, then the same
expression with std:: instead of qNN:: will compile and work the same
way. IOW: we can leave functionality out vis a vis std, but what
implement must be 100% compatible with the std version: no Qt-ish names,
no funny optimisations std doesn't have, etc.
And if std::foo changed in C++MM, MM > NN, and I've already implemented
qNN::foo, I'm not going to change the implementation in qNN, I create a
new implementation in qMM.
Conversely, as a user of these functions, when faced with implementation
of q14::exchange and q20::exchange, you should pick the one that has all
the features you need, not blindly pick the latest one. So far, the
higher-version version has a small comment that says what's changed:
// like C++20 is_sorted (ie. constexpr)
Finally, in order to check that there will be no problems when we make
the switch to a newer C++ version, each qNN function or type will
automatically be an alias to the std version, if the latter is available:
// like C++20 is_sorted (ie. constexpr)
namespace q20 {
#ifdef __cpp_lib_constexpr_algorithms
using std::is_sorted;
#else
~~~ our implementation ~~~
#endif
}
So when you make the switch from q20 to std, it's literally no change at
all.
On the downside, this means we can't use these types in public API,
which is why QSpan cannot be q20::span. Mostly, there's stuff useful for
implementation, though, so that's less of a restriction than it might seem.
There are quite a few components in namespace q20 already, and we even
have one that's not in the standard, yet: qxp::function_ref (xp for
eXPerimental).
All qNN headers contain this warning:
//
// W A R N I N G
// -------------
//
// This file is not part of the Qt API. Types and functions defined
// in this file will behave exactly as their std counterparts. You
// may use these definitions in your own code, but be aware that we
// will remove them once Qt depends on the C++ version that supports
// them in namespace std. There will be NO deprecation warning, the
// definitions will JUST go away.
//
// If you can't agree to these terms, don't use these definitions!
//
// We mean it.
//
Thanks for reading this far,
Marc
More information about the Development
mailing list