[Development] QtCS 2017 logging/tracing session notes

[Arnaud Clère]

Hi Christian,
Thanks for the pointers regarding LTT and the underlying Common Trace Format, that was very informative.
The fact that CTF supports variants and dynamically sized structures makes it a possible output for modmedLog, as is IETF's Compressed Binary Object Representation.

What is not possible is to use Bind<T> templates to generate a "static" TSDL schema of T, that is, without "runtime" variants. For instance a Bind<QSharedPointer<T>>::bind() will not call Bind<T>::bind() if null. So, the corresponding TSDL must use a variant to account for this possibility. More importantly, a Bind<QVector<T>> would be described as a sequence of variant of either : record | sequence | uint8_t | etc. Also a Bind<MyClass> would be described in the same way as a "generic" Bind<QStringMap<T>> putting all MyClass member names into the trace instead of describing them externally in the TSDL.

This less compact binary representation brings a lot of flexibility in that it would allow reusing existing qDebug tracepoints in the new tracing facility as long as the traced data has a Bind<T> defined (which would be the case for basic Qt types). On the other hand, the impact on performance should be reasonable. For instance, if added keys take 1/3 of your trace volume, I would expect the trace throughput to be 2/3 of the throughput of a trace fully described by a "static" TSDL.

Do you think the tradeoff would be acceptable?

I still need to figure out the best way to insert the tracepoint metadata from qDebug-like tracepoints into CTF. In our TSV+JSON format, we eliminate metadata redundancy on the fly so it does not take more space than actual event data. I can see that CTF has metadata packets for that, so it should be possible to do the same.

Regarding converting large traces to JSON, I agree it is not appropriate for your use case. The idea is to agree on a common logical data format based on JSON to allow supporting more physical formats (such as CTF) in the future. But I cannot tell when such support would be available.

Regarding access to forge.imag.fr repository. Did it eventually work? Otherwise, I'll mention that to the admin team.

Thanks a lot!
Arnaud