[Development] Important recent changes in QList/QString/QByteArray

Andrei Golubev andrei.golubev at qt.io
Mon Aug 31 14:01:05 CEST 2020


Hello all,


Over the last week I pushed several patches to enable prepend optimization in some core containers, namely, QList, QString, QByteArray.
In the light of these changes, there may occur new subtle issues in your code.


New model
---

In a nutshell, prepend optimization code does some magic with the memory layout and the stored elements location.
This, in turn, affects how you can and cannot use iterators/pointers/references to container's elements and their invalidation.

As of today, please consider that any non-const member function of the aforementioned container can invalidate "remembered" iterators, pointers or references to elements of that container. For example, an iterator is returned by QList::begin(), a pointer is returned by QString::data() (and QString::constData()), a reference is returned by QList::at() - this is, of course, not an exhaustive list.
The invalidation existed before for cases when a container could detach (due to copy-on-write) or reallocate (due to growing or squeezing). Now this is also true for non-detaching, non-reallocating, modifying operations. Examples of such operations: append, prepend, insert, emplace, remove, etc.


How this affects existing code?
---

To the best of my knowledge, at least certain documentation pieces of QString and QByteArray already explained such a thing. So, hopefully, diligent users of these 2 containers should be safe.

QList's behavior changed with regards to the iterators/pointers: as described above, using an iterator or a pointer to an element of a QList between non-const modifying operations is an undefined behavior (same true for references).

A more subtle issue concerns the users of QVector. Since QVector got unified with QList and QList's implementation/behavior changed, old code that was valid before may now be a subject to UB. Consider a simple example of the difference:

QVector<int> ints { 0, 1, 2, 3, 4, 5, ... };  // now: effectively QList<int>
auto thirdElemIt = ints.begin() + 2;
ints.remove(6);   // removing element at position 6. before: thirdElemIt remains valid
                           //                                                       now: thirdElemIt may be invalidated
print(*thirdElemIt);  // before: valid enough behavior; now: undefined


Aiding the debugging process
---

Below are some basic steps one can do to simplify the debugging of issues:

1) Enable assertions in the code. This should be possible even if you have a release build, there's a macro definition QT_FORCE_ASSERTS that should force-enable assertions (albeit not sure how exactly to specify it during build stage).

2) Build with address sanitizer. Even basic "sanitize=address" should suffice initially. It should be available both for GCC and Clang (guessing that any recent GCC/Clang that supports C++17 also supports ASan). MSVC developers some time ago also reported that ASan is now supported. To enable ASan locally with CMake build toolchain, one can use "-DFEATURE_sanitize_address=ON" flag as part of the CMake input arguments (when building Qt sources). Don't know an alternative if you use configure script + qmake.

3) Valgrind's memory checker should cover roughly same issues as address sanitizer.

Note that you do not need to rebuild Qt/your application to use valgrind which may be a preferred option sometimes.



Feel free to contact me in case you find an issue caused by new changes and need a help in resolving it.
Sorry for possible inconveniences!

--
Best Regards,
Andrei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20200831/5283bffa/attachment.html>


More information about the Development mailing list