[Development] Question about QCoreApplicationData::*_libpaths

[Kevin Kofler]

Bubke Marco wrote:
> Actually this convince is hurting you if you need performance. I would
> prefer the convenience on top of lower level api.
[and at the end:]
> It really depends what you want to do. I would prefer it we had a CoW
> wrapper around std vector. Best of both worlds.

The question is whether such a wrapper would be able to support all 
functionality of the current QVector or whether it would run into some 
limitations. I suspect we would lose at least some optimizations from 
Q_DECLARE_TYPEINFO. There might also be some API that cannot be implemented, 
or at least not efficiently. (This could be even worse for other containers 
than QVector.) So, it sounds like a great idea in theory, but I wonder how 
practical it is in practice. I guess I would have to see a practical 
implementation to know for sure.

What I know for sure is that my priority queue abstraction on top of 
std::priority_queue can support only a very limited API, and it would be the 
same if QQueue were to be implemented on top of std::queue: The current 
QQueue publicly inherits the underlying QList. The STL API, instead, wraps 
the underlying vector and only offers the queue primitives, a completely 
different philosophy that leads to a much more limited API. Of course, this 
could be worked around by wrapping std::vector directly (or maybe 
std::dequeue), or by simply keeping QQueue implemented on top of QList and 
only changing QList. But there might be other such API limitations in the 
STL.

And QList might not be implementable at all on top of the STL, at least not 
the flexible way it is now (array of pointers, except when the type can be 
put in the place of the pointer).

> I prefer algorithms which are based on iterators to handwritten loops.
> They are easier to parallise in the future too.

Well, then I have good news for you:
some_algorithm(myQVector.begin(), myQVector.end());
is perfectly safe. If myQVector was shared, the begin() or end() call, 
whichever is executed first, will detach, and then some_algorithm will 
operate on a new deep copy of myQVector that nothing else can possibly have 
seen yet. If myQVector was not shared to begin with, then it is of course 
safe too and there is no detach in that case.

The iterators are only problematic when you keep an iterator across an 
assignment, such as:
QVector<int>::iterator evil = myQVector.begin();
QVector<int> broken = myQVector;
*evil = 666;
where "evil" will corrupt "broken".

IMHO, this is simply a user error and a case of "don't do that then". This 
issue is explicitly warned about in the Qt documentation. If you really need 
a copy of myQVector at this place, then you can use this:
QVector<int>::iterator evil = myQVector.begin();
QVector<int> unbroken = myQVector;
unbroken.data(); // force detach
*evil = 666;
and everything will magically work again.

> Atomics on the other side can produce strange performance bugs with false
> sharing. I don't believe they are the future in a many core environment
> where you share cache lines very often.

Well, we could in principle do CoW without atomics (Qt 3 did them that way), 
but the drawback then is that we would lose thread-safety (which is of 
course why Qt 4 introduced atomic reference counts).

> Hmm most other languages I know provide  more convenience than Qt but are
> slower. I think you pick C++ in the context of speed. So we should provide
> a wrapper around std vector with cow.

Well, most other languages I know provide *less* convenience than Qt but are 
slower. ;-) See also my rant about Java elsewhere in this thread. Its 
"everything is a reference" explicit sharing (which I have also seen in many 
other languages, even and especially ones marketed to beginners: Visual 
Basic, Python and many more) is a poor substitute for CoW. That sharing 
issue you can run into when using iterators on Qt containers (see above) is 
how those languages *always* work: If you do not explicitly clone the 
object, writing to any "copy" will actually affect them *all*. And even 
Java's "final" will not protect you, because it is only the equivalent of a
Foo * const, not of a const Foo * or const Foo &. (That's why immutable 
objects are so popular in those programming languages, but those are of 
course a major performance issue if you need to make any modifications, 
because you are deep-copying all the time.)

> QList is really a trap.
> 
> struct Entry {
>   QString text;
>   bool isHtml;
> } ;
> 
> QList<Entry> list;
> 
> Do you see the problem?

I see that this is going to create a vector of Entry * and so require a lot 
of allocations of size sizeof(void *) + 1 + padding. Congratulations, you 
found the worst case of QList. Now imagine a future version of your code 
changes your Entry to look like this:
struct Entry {
  QString text;
  bool isHtml;
  int data[10000];
} ;
and suddenly, the QList strategy will not be that bad an idea anymore, 
especially if you find yourself doing random inserts or removals. 
(Incidentally, they plan to change QString in Qt 6 with an alleged 
"optimization" that will actually make your struct look not all that 
different from that: They want to make QString big so that they can store 
small strings without an allocation. I call that the "small string 
pessimization". I strongly doubt the saved allocation is worth passing 
around huge blobs for a type as common as QString.) Good luck waiting for 
the memory move of your QVector or std::vector if you delete element 5000 
out of a 10000-element vector of this new Entry type.

        Kevin Kofler