[Development] QTBUG-43096 - QML instantiation performance decadence

[Chris Adams]

On Thu, May 31, 2018 at 5:21 PM, Uwe Rathmann <Uwe.Rathmann at tigertal.de> wrote:
> Hi Simon,
>
>> I ran benchmarks comparing a release build of 5.12 against 5.1.1 and ran
>> the benchmark mentioned in the task, where Qt 5.12 came out in average
>> faster by a factor of 4.
>
> Christophers comment in the JIRA ticket raises some questions concerning
> the correctness of those benchmarks.
>
> Would you be so kind to clarify how far we can trust in these numbers ?

I did have a couple of questions about that particular benchmark
result, but that doesn't mean that the conclusion is incorrect.

It's important to note that the issue may lie in the original numbers,
as the library metrics test previously used some hand-rolled methods
to discard outliers and perform an average, and worked on wall time
rather than cpu ticks - a method which may very well have been wrong
(and that would be entirely my fault).

Also, it is worth pointing out that recently a considerable effort has
been made to improve the benchmarking of all stages of QtQuick
applications (from the JS side of things, to the QML compiler side of
things, to the scene graph side of things), as the grafana and
qmlbench etc tracking proves.  Performance regressions are now being
caught, and performance metrics are easily visible, thanks to those
efforts, and that is definitely have a very positive effect on
performance outcomes for everyone.

FWIW I am certain that the QML engine in Qt 5.12 will indeed perform
better than it did in Qt 5.1 days (or any other version of Qt) because
the people working most closely with the engine and the people working
most closely with QtQuick both believe it to be so, and I trust their
judgement.

Best regards,
Chris.