[Interest] Faster QXmlStreamWriter?

[Thiago Macieira]

On Tuesday, 24 April 2018 09:39:37 PDT Julien Cugnière wrote:
> 2018-04-24 17:52 GMT+02:00 Thiago Macieira <thiago.macieira at intel.com>:
> > On Tuesday, 24 April 2018 07:39:08 PDT Konstantin Tokarev wrote:
> >> If your serialized data is intended to be read by Qt applications only,
> >> QDataStream may be a good choice
> > 
> > Also slow.
> > 
> > Binary QJsonDocument is the fastest, followed closely by QCborValue (new
> > in
> > 5.12).
> 
> Out of curiosity, what makes QDataStream slow? From your other comment
> on QBuffer, it sounds like it might be related to the QIODevice
> interface. Does that mean it's impossible to get best performance
> through QIODevice, because of some design flaw? Or is there something
> else that makes QDataStream and QBuffer slow?

I benchmarked the full result, but didn't investigate what makes QDataStream 
slow. But it suffers from the same problem that QXmlStreamWriter does: it uses 
a QBuffer to write to a QDataStream (see [1] and [2]). QCborStreamWriter does 
the same, actually -- I've only fixed QCborStreamReader to bypass it.

QDataStream's format is also quite big, storing all QStrings as UTF-16 with a 
4-byte length prefix. So "Hello World" takes 4 + 2*11 = 26 bytes, whereas in 
CBOR it takes 1 + 11. The larger format could mean we reallocate more often 
and that causes delays.

Another issue is that QDataStream tries to store all integers in big endian, 
unless you tell it otherwise. Since most machines are little-endian, that's a 
waste of CPU cycles, as you're doing the endian conversion twice, for little 
gain, however fast it is. 

Finally, yes, QIODevice is not designed for performance. It's quite slow and 
there's little we can do about it. There have been a lot of behind-the-scenes 
improvements, by way of smarter buffering and sharing of more code between the 
main QIODevice classes (QFile, QProcess, QTcpSocket, QNetworkReply 
implementations). QBuffer is usually forgotten in that. Still, the main 
problem is QIODevice's *design* and we're unable to change it.

[1] https://code.woboq.org/qt5/qtbase/src/corelib/serialization/
qdatastream.cpp.html#_ZN11QDataStreamC1EP10QByteArray6QFlagsIN9QIODevice12OpenModeFlagEE
[2] https://code.woboq.org/qt5/qtbase/src/corelib/serialization/
qdatastream.cpp.html#_ZN11QDataStreamC1ERK10QByteArray

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center