[Interest] Faster QXmlStreamWriter?

[Thiago Macieira]

On Wednesday, 25 April 2018 08:45:41 PDT Thiago Macieira wrote:
> On Wednesday, 25 April 2018 07:13:18 PDT Konstantin Tokarev wrote:
> > 25.04.2018, 09:43, "Thiago Macieira" <thiago.macieira at intel.com>:
> > > So Qt 6 QJsonDocument & family will have the 128 MB limit
> > > removed, at the expense of the binary JSON format requiring parsing.
> > 
> > That's sad, as we lose the single solution inside Qt for serialization
> > without parsing
> 
> That was going to happen anyway, because of the 128 MB size limit. If we did
> nothing else, we'd create a new format with a much expanded size limit,
> which means the current format would need to be parsed and converted.

By the way, QJsonDocument::fromBinaryData does still perform a correctness 
check, to make sure it won't crash later reading corrupt data. You can skip 
this step and then loading your data is extremely fast.

Here are my numbers comparing loading that 60+ MB file in both binary JSON 
format (with validation) and CBOR:

Binary JSON:
         69,844846      task-clock:u (msec)
       196.906.259      cycles:u
       422.255.714      instructions:u
[There's no readAll(); 70.2% of the time is spent inside 
QJsonPrivate::Object::isValid]

JSON:
        255,809132      task-clock:u (msec)
       771.771.000      cycles:u
     2.690.966.058      instructions:u
[80.2% inside QJsonPrivate::Parser::parseValue, 58.7% inside 
QJsonPrivate::Parser::parseString and 16.3% inside QUtf8Functions::fromUtf8]

CBOR:
        239,059121      task-clock:u (msec)
       562.474.857      cycles:u
     1.431.590.428      instructions:u
[71.6% inside QCborValue::fromCbor, 65.0% inside 
QCborContainerPrivate::decodeStringFromCbor, 25.5% inside 
QCborStreamReader::readStringChunk plus 12.6% inside QUtf8::isValidUtf8]

So it's just under 4x slower, but we're still talking about consuming over 
250 MB/s of data.

PS: YMMV, especially if you don't use CPU-optimised UTF-8 methods like I do. 
You need to compile your own Qt to get those.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center