[Interest] json file handling

[Andre Somers]

On 15-4-2015 17:25, Thiago Macieira wrote:
> On Wednesday 15 April 2015 16:36:15 André Somers wrote:
>>> The dilemma I'm in is, how can I tell the json writer to "append" to a
>>> current json file without loading the whole file back in memory (eg
>>> read json from file->edit in Qt->write back to file). Because, after a
>>> while, I suppose the file will get quite large.
>> I don't think JSON is particulary suitable for this kind of application
>> (and neither is XML).
> CBOR might be, since it does support "undefined length" maps and arrays.
I'm not familiar with that format, but I'll look it up. I guess the 
choice for JSON or XML is made due to convenience though, as classes to 
read and write these are readily available. I doubt the same can be said 
of CBOR.
> As for XML, the format may not be suitable but QXmlStreamWriter is since it
> keeps the state of which tags it needs to close and allows you to simply
> stream data into it. On the receiver side, QXmlStreamReader is capable of
> receiving incomplete data and making the best it can with what has been
> completed.
I'd expect a log-file like output to be well defined at any moment in 
time, or as close as possible to that. So, while QXmlStreamWriter is 
suitable, it does not produce a valid XML file until you finalize the 
write. That may not be what you want (even though QXmlStreamReader could 
be used to parse the incomplete file*).

I think a simpler aproach may be a better one in such cases: simply add 
new values to the end of a file when the become available. No fancy 
markup languages to wrap them. That will result in a smaller file 
(especially compared to XML), faster writing and no problems reading the 
data back in. Simple CSV will do if you want a text-based file, but a 
binary file could also function really well if that is not needed. Just 
append to the file when you have new data available. Such approaches 
have worked with log files for decades :-)

André

*) Though I seem to remember there are actually some problems with that 
in reality.