[Development] RFC: unified data model API in QtCore

[Lars Knoll]

> On 14 Jun 2018, at 02:08, Thiago Macieira <thiago.macieira at intel.com> wrote:
> 
> Out of the serialisation discussion at QtCS 2018, we have a call for action to 
> discuss the possibility of *not* adding QCborValue in Qt 5.12 and instead add 
> a generic, data model API that could be used for JSON, CBOR and future uses. 
> The reason was that we do not need the duplication of very similar-looking 
> APIs, with the duplication of documentation that goes along with it. Other 
> languages parse serialised data into their own data structures which provide a 
> generic access and modification API.
> 
> This email is to start that discussion and answer these questions:
> 
> 	a) should we have this API?
> 
> 	b) if so, what would this API look like?
> 
> 	c) if not, should we unify at least JSON and CBOR?
> 
> 	c) either way, should remove QCborValue until we have it?
> 
> The current "Qt native" data structure is the trio QVariant, QVariantList and 
> QVariantMap. It is what some early (pre-Qt5) JSON parsers used, as JSON data 
> can be losslessly converted to a handful of metatypes. And as long as only 
> those types were used, the conversion to JSON is also lossless.

I guess we all agree that QVariant is not really a good API for this :)
> 
> This API would also be the replacement of the JSON and CBOR parsers, for which 
> we'd have a unified API, something like:
> 
> 	QFoo json = QJson::fromJson(jcontent);
> 	QFoo cbor = QCbor::fromCbor(ccontent);
> 	qDebug() << QCbor::toDiagnosticFormat(json);	// yes, json
> 
> And here's why I am having problems with this proposal: what else? We know we 
> can have a common API for JSON and CBOR because I've done it (it's QCborValue) 
> and because CBOR is designed to be compatible with JSON. But what about other 
> protocols? Are we trying to shoehorn a square peg in a round hole? Recreate 
> multidimensional hierarchical tables (a.k.a., QAbstractItemModel))?

I agree that we should avoid something like QAIM. What we can (and IMO should) do is share the implementation of the data model between CBOR and JSON.
> 
> For example, the next data model over would be XML. And we are indeed missing 
> a DOM structure for it since we deprecated QDomDocument. Can anyone see a data 
> structure that works for both JSON/CBOR and generic XML? More importantly, one 
> that is worthy of being Qt, with nice, intuitive API?

I’d leave XML out of this. It is difficult to integrate, as it has so many special features (entities, CDATA and lot of other things) making it a rather complex specification. But there are a couple of other formats that might fit a more generic data model. Yaml comes to my mind as an example, protobuf and flatbuffers might also be possible to stream into such a structure with some additional schema verification.
> 
> I'm skeptical. What do you think?f

One option could also be that we have a common implementation, and then a very thin API wrapper for each of the formats on top that will enforce the format specific limitations and give you a fully typed API.

Cheers,
Lars