[Interest] Faster QXmlStreamWriter?

[Thiago Macieira]

On Tuesday, 24 April 2018 22:46:43 PDT Maurice Kalinowski wrote:
> > On Tuesday, 24 April 2018 09:39:37 PDT Julien Cugnière wrote:
> > > 2018-04-24 17:52 GMT+02:00 Thiago Macieira <thiago.macieira at intel.com>:
> > > > On Tuesday, 24 April 2018 07:39:08 PDT Konstantin Tokarev wrote:
> > > >> If your serialized data is intended to be read by Qt applications
> > > >> only, QDataStream may be a good choice
> > > > 
> > > > Also slow.
> > > > 
> > > > Binary QJsonDocument is the fastest, followed closely by QCborValue
> > > > (new in 5.12).
> > > 
> > > Out of curiosity, what makes QDataStream slow? From your other
> > 
> > comment
> > 
> > > on QBuffer, it sounds like it might be related to the QIODevice
> > > interface. Does that mean it's impossible to get best performance
> > > through QIODevice, because of some design flaw? Or is there something
> > > else that makes QDataStream and QBuffer slow?
> > 
> > I benchmarked the full result, but didn't investigate what makes
> > QDataStream slow. But it suffers from the same problem that
> > QXmlStreamWriter does: it uses a QBuffer to write to a QDataStream (see
> > [1]
> > and [2]). QCborStreamWriter does the same, actually -- I've only fixed
> > QCborStreamReader to bypass it.
> 
> [Maurice Kalinowski]
> 
> Hey,
> 
> Would you still have these benchmarks somewhere? I am currently playing
> around with various serialization ways as well, and so far came to
> different conclusions. Probably because of using different means, but would
> still like to see how you approached it for plain text json.

I posted to dev in the CBOR thread a few months ago. Basically, the procedure 
was:

 1) find a large-ish JSON file as a test seed
 2) make it bigger (replicate it 100x or more in an array) -- I used something 
   like 60 MB in binary JSON form, which is just about half the maximum
 3) use the examples/corelib/serialization/convert tool[1] to convert to other 
   formats
 4) use the same tool with the "null" output to benchmark reading[2]
 5) use the tool with binary JSON as source to benchmark writing

I basically benchmarked using Linux's perf. The stat subcommand for overall 
timings, but using perf record + perf annotate to analyse where the issues 
were.

[1] https://codereview.qt-project.org/217410. The tool should work for all 
other formats right now if you just remove "cborconverter.cpp" from the .pro 
file.

[2] I modified main.cpp to read and write using QBuffer instead of QFile to 
minimise the cost of QIODevice.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center