[Development] QDataStream: blackbox or document all versions?

[Konstantin Tokarev]

26.09.2016, 12:35, "Simon Hausmann" <simon.hausmann at qt.io>:
> Hi,
>
> I'm very much in favor of using a proper schema based system such as protocol buffers if we decide
>
> to remove the black box from serialization. They don't appear to be connected, but the moment you
>
> need to deal with changes in the format, the protobuf approach wins IMO. The other advantage is interoperability
>
> with the world outside of Qt.

Protobuf itself is kinda outdated format, there are a lot of newer alternatives which incur much lower overhead. But it certainly has advantage of wider interoperability.

>
> But this isn't so much a question of what to do, I I think it's more of a question of man power. Until then I'm in favor
>
> of QDataStream remaining a black box.
>
> Simon
> ----------------------------------------
> From: Development <development-bounces+simon.hausmann=qt.io at qt-project.org> on behalf of Michael Winkelmann <michael.winkelmann at qt.io>
> Sent: Monday, September 26, 2016 10:40:02 AM
> To: development at qt-project.org
> Subject: Re: [Development] QDataStream: blackbox or document all versions?
>
> I think what Qt is lacking is a good serialization module like cereal or
> boost serialization.
> With this pattern, QDataStream would be just another archive format,
> like JSON or XML.
> Cereal has portable and unportable binary for endianness.
>
> The developer should be able specify how the data is serialized and
> select an arbitrary archive type that takes streams operators as default
> serialization method.
>
> PS: For wire protocols, I would also consider Protobuf as an option.
>
> On 26.09.2016 10:11, Ulf Hermann wrote:
>> On 09/25/2016 05:58 AM, Thiago Macieira wrote:
>>> A thread[1] on the interest mailing list started when someone asked
>>> for the
>>> docs for the current format of the QDataStream wire protocol, to which I
>>> replied that it doesn't exist as we don't maintain such docs.
>>>
>>> Long story short, we have two options: [...]
>>
>> I see another option: Make QDataStream a good interchange format.
>> From your reply on the interest mailing list:
>>
>>> I also don't think the QDataStream binary format is particularly good
>>> for non-
>>> Qt purposes. It passes internals that are really about how Qt works,
>>> not meant
>>> for interchange of information. For example, QDate passes the Julian
>>> Day in
>>> 64-bit format -- who in the world exchanges data like that? One would
>>> expect
>>> the serialisation format of a date and time to be something like
>>> milliseconds
>>> since 1970, milliseconds since 1601 or a string format, like ISO
>>> 3337. It
>>> passes strings as UTF-16 encoded instead of encoding with UTF-8,
>>> which would
>>> in turn allow interchanging QString and QByteArray serialisation
>>> formats.
>>>
>>> It's also by default quite inefficient since it marshalls everything
>>> as big-
>>> endian, while most CPUs are little-endian. It has no support for
>>> indexing or
>>> random-access seeking. It can't be used as a DOM storage, like
>>> QJsonDocument
>>> can.
>>
>> All of that, except for the default endianness, could be fixed on a
>> per operator base. Changing the default endianness can also be done
>> with minimal effort and would only require incrementing the data
>> stream version. Considering this, QDataStream may at the moment not be
>> very efficient, but it does have some potential. We could probably
>> even change all the operators to read and write CBOR, and that would
>> also only require a version change. The versioning allows us to adapt
>> the wire format to whatever is needed now or in the future. I don't
>> know any other data interchange format which can do this.
>>
>> This doesn't change the implicit conversion problem that Marc
>> mentioned, but at the moment I don't quite see a way to fix it without
>> removing a lot of convenience for the user (that is, have the user
>> specify the exact intended type for each serialization operation).
>>
>> regards,
>> Ulf
>>
>>
>>
>> _______________________________________________
>> Development mailing list
>> Development at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
> ,
>
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development


-- 
Regards,
Konstantin