[Development] QML instantiation performance
Chris Adams
chris.adams at qinetic.com.au
Mon Dec 1 06:01:28 CET 2014
Hi Gunnar,
Note that the results between versions may not be directly comparable.
The benchmark does some very specific things, including destroying the
engine and clearing all type registrations between each run. This is
intended to simulate "application startup performance" conditions, but is
inaccurate even so (since during application startup, multiple components
will be loaded, however only the very first component would be loaded in
the environment that this benchmark simulates).
For example, consider a component with some bindings (or other JS
expressions which get run during instantiation). In this case, in the
simulated clean environment, the QML engine will have to create a root JS
context, populate it with various context objects (the Qt global object,
etc), build a scope chain for the component context, compile the
expressions and finally evaluate them, for every single run of the
benchmark. In a real application, this price is paid exactly once, and
subsequent components will require only their expressions to be compiled
and evaluated.
If the V4 JIT is vastly more performant (due to the better knowledge about
QML types, for example) once instantiated, but takes a lot longer to
instantiate, then this benchmark will vastly overstate the problem (since
that startup cost is reported for every data row), and understate the
benefits (since each QML file in the test set is individually small and
don't include many bindings / expressions; whereas a real application will
have many).
So, just a word of warning when interpreting the results :-) More
benchmarks would have to be written which compare "cached / warmed-up"
performance between the different versions of Qt, to get the full picture.
Cheers,
Chris.
www.qinetic.com.au - Qt And QML User Experience Specialists
On Sat, Nov 29, 2014 at 2:11 AM, Gunnar Roth <gunnar.roth at gmx.de> wrote:
>
> Hi.
> on windows that is even worse as
> the qtdeclarative/tests/benchmarks/qml/librarymetrics_performance
> benchmark is crashing in many testcases. 5.2.1 runs some more cases than
> 5.3.2 and 5.4 beta.
>
> the test cases which crash in 5.3.2 and 5.4 beta are:
> //QTest::newRow("039) listView - with anchors and list model content with
> delegate") << testFileUrl("data/listview.3.qml");
> // QTest::newRow("046) image - with content, async") <<
> testFileUrl("data/image.4.qml");
> // QTest::newRow("053) repeater - row - repeat simple rectangle") <<
> testFileUrl("data/repeater.1.qml");
> // QTest::newRow("054) repeater - column - repeat simple rectangle") <<
> testFileUrl("data/repeater.2.qml");
> // QTest::newRow("058) positioning - anchored positioning") <<
> testFileUrl("data/anchoredpositioning.qml");
> // QTest::newRow("059) positioning - anchored (with grid) positioning")
> << testFileUrl("data/anchorwithgridpositioning.qml");
> // QTest::newRow("061) positioning - anchored (with binding) positioning")
> << testFileUrl("data/anchorwithbindingpositioning.qml");
>
>
> // QTest::newRow("03) positioning - anchored positioning") <<
> testFileUrl("data/anchoredpositioning.2.qml");
> // QTest::newRow("04) positioning - anchored (with grid) positioning") <<
> testFileUrl("data/anchorwithgridpositioning.2.qml");
>
>
> // QTest::newRow("06) positioning - anchored (with binding) positioning")
> << testFileUrl("data/anchorwithbindingpositioning.2.qml");
> // QTest::newRow("07) positioning - binding (with grid) positioning") <<
> testFileUrl("data/bindingwithgridpositioning.2.qml");
>
>
>
>
> I tried to let it run with 5.1 on windows but i am getting lots of weird
> warnings i dont understand:
>
> Config: Using QtTest library 5.1.0, Qt 5.1.0
> PASS : tst_librarymetrics_performance::initTestCase()
> QWARN : tst_librarymetrics_performance::compilation(001) item - empty)
> ERROR: (file:///data/item.1.qml:3:1: Item is not a type
> Item {
> ^)
>
>
> where item.1.qml is just
> importQtQuick2.0
>
> Item{
> }
>
>
> i also get this for other types in the other tests. Furthermore i get:
> QWARN : tst_librarymetrics_performance::instantiation_cached(007) item -
> uninitialised Item prop) QQmlComponent: Component is not ready
> a lot.
>
>
> When looking at the all the perf numbers for 5.2.1 and 5.3.2, i can see
> that a lot are slower in 5.3.2 but only a few faster . Comparing 5.3.2
> with 5.4 some are slower some are faster.
>
> as for 5.1.0 , getting most of the time errors i dont count the result as
> reliable.
>
>
> As the positioner tests work for all, so here are these results
>
>
> Qt 5.1.0 has
> PASS : tst_librarymetrics_performance::instantiation(056) positioning -
> no positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"056) positioning
> - no positioning":
> 2,936,250 nsecs
> RESULT : tst_librarymetrics_performance::instantiation():"056) positioning
> - no positioning":
> 2,936,250 nsecs
> PASS : tst_librarymetrics_performance::instantiation(057) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"057) positioning
> - absolute positioning":
> 3,101,814 nsecs
> RESULT : tst_librarymetrics_performance::instantiation():"057) positioning
> - absolute positioning":
> 3,101,814 nsecs
> PASS : tst_librarymetrics_performance::instantiation(060) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"060) positioning
> - binding positioning":
> 3,054,007 nsecs
> RESULT : tst_librarymetrics_performance::instantiation():"060) positioning
> - binding positioning":
> 3,054,007 nsecs
> PASS : tst_librarymetrics_performance::instantiation(062) positioning -
> binding (with grid) positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"062) positioning
> - binding (with grid) positioning":
> 2,984,185 nsecs
> RESULT : tst_librarymetrics_performance::instantiation():"062) positioning
> - binding (with grid) positioning":
> 2,984,185 nsecs
> PASS : tst_librarymetrics_performance::positioners(01) positioning - no
> positioning)
> RESULT : tst_librarymetrics_performance::positioners():"01) positioning -
> no positioning":
> 2,958,458 nsecs
> RESULT : tst_librarymetrics_performance::positioners():"01) positioning -
> no positioning":
> 2,958,458 nsecs
> PASS : tst_librarymetrics_performance::positioners(02) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::positioners():"02) positioning -
> absolute positioning":
> 3,096,950 nsecs
> RESULT : tst_librarymetrics_performance::positioners():"02) positioning -
> absolute positioning":
> 3,096,950 nsecs
> PASS : tst_librarymetrics_performance::positioners(05) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::positioners():"05) positioning -
> binding positioning":
> 3,085,942 nsecs
>
> Qt 5.2.1 has much slower results:
> PASS : tst_librarymetrics_performance::instantiation(056) positioning -
> no positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"056) positioning
> - no positioning":
> 9,190,951 nsecs
> PASS : tst_librarymetrics_performance::instantiation(057) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"057) positioning
> - absolute positioning":
> 9,904,790 nsecs
> PASS : tst_librarymetrics_performance::instantiation(060) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"060) positioning
> - binding positioning":
> 11,098,362 nsecs
> PASS : tst_librarymetrics_performance::instantiation(062) positioning -
> binding (with grid) positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"062) positioning
> - binding (with grid) positioning":
> 11,427,314 nsecs
> PASS : tst_librarymetrics_performance::positioners(01) positioning - no
> positioning)
> RESULT : tst_librarymetrics_performance::positioners():"01) positioning -
> no positioning":
> 13,258,183 nsecs
> PASS : tst_librarymetrics_performance::positioners(02) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::positioners():"02) positioning -
> absolute positioning":
> 52,944,352 nsecs
> PASS : tst_librarymetrics_performance::positioners(05) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::positioners():"05) positioning -
> binding positioning":
> 117,415,282 nsecs
>
>
> Qt 5.3.2 is even slower:
> PASS : tst_librarymetrics_performance::instantiation(056) positioning -
> no positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"056) positioning
> - no positioning":
> 9,141,288 nsecs
> PASS : tst_librarymetrics_performance::instantiation(057) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"057) positioning
> - absolute positioning":
> 10,591,046 nsecs
> PASS : tst_librarymetrics_performance::instantiation(060) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"060) positioning
> - binding positioning":
> 12,113,250 nsecs
> PASS : tst_librarymetrics_performance::instantiation(062) positioning -
> binding (with grid) positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"062) positioning
> - binding (with grid) positioning":
> 12,440,730 nsecs
> PASS : tst_librarymetrics_performance::positioners(01) positioning - no
> positioning)
> RESULT : tst_librarymetrics_performance::positioners():"01) positioning -
> no positioning":
> 12,882,640 nsecs
> PASS : tst_librarymetrics_performance::positioners(02) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::positioners():"02) positioning -
> absolute positioning":
> 53,370,390 nsecs
> PASS : tst_librarymetrics_performance::positioners(05) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::positioners():"05) positioning -
> binding positioning":
> 168,676,730 nsecs
>
>
> And Qt 5.4 is even more slower:
> PASS : tst_librarymetrics_performance::instantiation(056) positioning -
> no positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"056) positioning
> - no positioning":
> 9,970,004 nsecs
> PASS : tst_librarymetrics_performance::instantiation(057) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"057) positioning
> - absolute positioning":
> 10,513,544 nsecs
> PASS : tst_librarymetrics_performance::instantiation(060) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"060) positioning
> - binding positioning":
> 12,005,796 nsecs
> PASS : tst_librarymetrics_performance::instantiation(062) positioning -
> binding (with grid) positioning)
> RESULT : tst_librarymetrics_performance::instantiation():"062) positioning
> - binding (with grid) positioning":
> 11,615,469 nsecs
> PASS : tst_librarymetrics_performance::positioners(01) positioning - no
> positioning)
> RESULT : tst_librarymetrics_performance::positioners():"01) positioning -
> no positioning":
> 16,094,852 nsecs
> PASS : tst_librarymetrics_performance::positioners(02) positioning -
> absolute positioning)
> RESULT : tst_librarymetrics_performance::positioners():"02) positioning -
> absolute positioning":
> 59,727,488 nsecs
> PASS : tst_librarymetrics_performance::positioners(05) positioning -
> binding positioning)
> RESULT : tst_librarymetrics_performance::positioners():"05) positioning -
> binding positioning":
> 208,747,723 nsecs
>
> Regards,
> Gunnar
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20141201/1dac8620/attachment.html>
More information about the Development
mailing list