[Development] QDataStream: blackbox or document all versions?

[Thiago Macieira]

On segunda-feira, 26 de setembro de 2016 10:29:07 PDT Shawn Rutledge wrote:
> On Sep 26, 2016, at 11:34, Simon Hausmann
> <Simon.Hausmann at qt.io<mailto:Simon.Hausmann at qt.io>> wrote:
> 
> Hi,
> 
> I'm very much in favor of using a proper schema based system such as
> protocol buffers if we decide to remove the black box from serialization.
> They don't appear to be connected, but the moment you need to deal with
> changes in the format, the protobuf approach wins IMO. The other advantage
> is interoperability with the world outside of Qt.

Interoperability would be a plus, indeed. But if you want Protobuf, why not 
use Protobuf? Why do you need QDataStream to do it?

> There are many alternatives to protocol buffers, some of them faster and/or
> more compressed.  https://capnproto.org/ claims to be a really good one,
> for example.  It has an MIT license.  Zero-copy is a nice feature to have. 
> After what I read, I’d probably never choose to use protocol buffers if it
> can be avoided.  You have to use the code generator, and it’s not
> efficient.  But I never tried, either.

Zero-copy is useless for Qt due to the way our strings are stored in memory 
anyway (UTF-16 host-endian) and due to our implicit sharing.

If I were to choose and implement this, I'd choose CBOR for these reasons:
* I personally already know the format
* I maintain a library to encode and decode it (whose unit tests are written 
   with QtTest)
* it's in an RFC (7049), which none other besides JSON is
* it's got its own MIME type (application/cbor) and CoAP content-format (60)

> Being able to mmap the file and immediately treat it as the data structure
> that you really wanted is a nice feature to have; that kind of
> implementation is not necessarily the same thing as a serialization
> protocol, although the ideal serialization protocol could be designed so
> that you can deal with it either way.  Doesn’t fit the QDataStream API,
> anyway.

Again, not so useful to us. If we need that, we already have one: the 
QJsonDocument binary format. If we need another, I'd recommend the dconf 
format, which is the same as the D-Bus wire format plus indexing, is already 
very much in use in Linux and has a Qt implementation already written.

And as you said, mmap() ability is usually at odds with streaming.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center