[Interest] Faster QXmlStreamWriter?
thiago.macieira at intel.com
Wed Apr 25 08:36:15 CEST 2018
On Tuesday, 24 April 2018 22:46:43 PDT Maurice Kalinowski wrote:
> > On Tuesday, 24 April 2018 09:39:37 PDT Julien Cugnière wrote:
> > > 2018-04-24 17:52 GMT+02:00 Thiago Macieira <thiago.macieira at intel.com>:
> > > > On Tuesday, 24 April 2018 07:39:08 PDT Konstantin Tokarev wrote:
> > > >> If your serialized data is intended to be read by Qt applications
> > > >> only, QDataStream may be a good choice
> > > >
> > > > Also slow.
> > > >
> > > > Binary QJsonDocument is the fastest, followed closely by QCborValue
> > > > (new in 5.12).
> > >
> > > Out of curiosity, what makes QDataStream slow? From your other
> > comment
> > > on QBuffer, it sounds like it might be related to the QIODevice
> > > interface. Does that mean it's impossible to get best performance
> > > through QIODevice, because of some design flaw? Or is there something
> > > else that makes QDataStream and QBuffer slow?
> > I benchmarked the full result, but didn't investigate what makes
> > QDataStream slow. But it suffers from the same problem that
> > QXmlStreamWriter does: it uses a QBuffer to write to a QDataStream (see
> > 
> > and ). QCborStreamWriter does the same, actually -- I've only fixed
> > QCborStreamReader to bypass it.
> [Maurice Kalinowski]
> Would you still have these benchmarks somewhere? I am currently playing
> around with various serialization ways as well, and so far came to
> different conclusions. Probably because of using different means, but would
> still like to see how you approached it for plain text json.
I posted to dev in the CBOR thread a few months ago. Basically, the procedure
1) find a large-ish JSON file as a test seed
2) make it bigger (replicate it 100x or more in an array) -- I used something
like 60 MB in binary JSON form, which is just about half the maximum
3) use the examples/corelib/serialization/convert tool to convert to other
4) use the same tool with the "null" output to benchmark reading
5) use the tool with binary JSON as source to benchmark writing
I basically benchmarked using Linux's perf. The stat subcommand for overall
timings, but using perf record + perf annotate to analyse where the issues
 https://codereview.qt-project.org/217410. The tool should work for all
other formats right now if you just remove "cborconverter.cpp" from the .pro
 I modified main.cpp to read and write using QBuffer instead of QFile to
minimise the cost of QIODevice.
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
More information about the Interest