[Interest] Faster QXmlStreamWriter?
annulen at yandex.ru
Wed Apr 25 18:55:55 CEST 2018
25.04.2018, 19:08, "Thiago Macieira" <thiago.macieira at intel.com>:
> On Wednesday, 25 April 2018 08:45:41 PDT Thiago Macieira wrote:
>> On Wednesday, 25 April 2018 07:13:18 PDT Konstantin Tokarev wrote:
>> > 25.04.2018, 09:43, "Thiago Macieira" <thiago.macieira at intel.com>:
>> > > So Qt 6 QJsonDocument & family will have the 128 MB limit
>> > > removed, at the expense of the binary JSON format requiring parsing.
>> > That's sad, as we lose the single solution inside Qt for serialization
>> > without parsing
>> That was going to happen anyway, because of the 128 MB size limit. If we did
>> nothing else, we'd create a new format with a much expanded size limit,
>> which means the current format would need to be parsed and converted.
> By the way, QJsonDocument::fromBinaryData does still perform a correctness
> check, to make sure it won't crash later reading corrupt data. You can skip
> this step and then loading your data is extremely fast.
For example, QtWebChannel implementations in QtWebKit and QtWebEngine
use QJsonDocument as a wire format for exchanging data between processes,
of course validation is skept because producer and consumer are reliable.
> Here are my numbers comparing loading that 60+ MB file in both binary JSON
> format (with validation) and CBOR:
> Binary JSON:
> 69,844846 task-clock:u (msec)
> 196.906.259 cycles:u
> 422.255.714 instructions:u
> [There's no readAll(); 70.2% of the time is spent inside
> 255,809132 task-clock:u (msec)
> 771.771.000 cycles:u
> 2.690.966.058 instructions:u
> [80.2% inside QJsonPrivate::Parser::parseValue, 58.7% inside
> QJsonPrivate::Parser::parseString and 16.3% inside QUtf8Functions::fromUtf8]
> 239,059121 task-clock:u (msec)
> 562.474.857 cycles:u
> 1.431.590.428 instructions:u
> [71.6% inside QCborValue::fromCbor, 65.0% inside
> QCborContainerPrivate::decodeStringFromCbor, 25.5% inside
> QCborStreamReader::readStringChunk plus 12.6% inside QUtf8::isValidUtf8]
> So it's just under 4x slower, but we're still talking about consuming over
> 250 MB/s of data.
> PS: YMMV, especially if you don't use CPU-optimised UTF-8 methods like I do.
> You need to compile your own Qt to get those.
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
> Interest mailing list
> Interest at qt-project.org
More information about the Interest