[Development] optimizing QComposeInputContext / TableGenerator?

Mon Jan 19 20:15:22 CET 2015

Hello all,

when I run my heaptrack [1] tool on Qt 5 applications, I often stumble upon 
the compose TableGenerator. It initializes many QStrings and also consumes 
ruoghly 400KB of memory. I wonder whether we could optimize this somehow?

The best approach of course would be to have a OpenDesktop standard that 
allows mmapping the compose table in and using it from there. Probably not 
feasible. Creating our own cache brings the usual issues of having to update 
the cache when the original changes... So what I wonder about is whether one 
could delay the table generation? I usually don't use the compose key, so my 
naive assumption would be that lazy-loading this table would help the common 
case of startup quite a bit already. Or is this required for other things that 
I don't expect?

5786 calls to allocation functions from:
QArrayData::allocate(unsigned long, unsigned long, unsigned long, QFlags<>)
  at sources/qtbase/src/corelib/tools/qarraydata.cpp:105
  in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QTypedArrayData<>::allocate(unsigned long, QFlags<>)
      at ../../include/QtCore/../../../src/corelib/tools/qarraydata.h:217
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QString::QString(int, Qt::Initialization)
      at sources/qtbase/src/corelib/tools/qstring.cpp:1497
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QUtf8::convertToUnicode(char const*, int, QTextCodec::ConverterState*)
      at sources/qtbase/src/corelib/codecs/qutfcodec.cpp:316
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QUtf8Codec::convertToUnicode(char const*, int, 
QTextCodec::ConverterState*) const
      at sources/qtbase/src/corelib/codecs/qutfcodec.cpp:671
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QTextCodec::toUnicode(char const*, int, QTextCodec::ConverterState*) const
      at ../../include/QtCore/../../../src/corelib/codecs/qtextcodec.h:103
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QString::fromLocal8Bit_helper(char const*, int)
      at sources/qtbase/src/corelib/tools/qstring.cpp:4554
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QString::fromLocal8Bit(char const*, int)
      at ../../../../include/QtCore/../../../src/corelib/tools/qstring.h:533
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::parseKeySequence(char*)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:402
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::parseComposeFile(QFile*)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:293
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::processFile(QString)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:267
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::findComposeFile()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:112
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::TableGenerator()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:56
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QComposeInputContext::QComposeInputContext()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/qcomposeplatforminputcontext.cpp:85
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QComposePlatformInputContextPlugin::create(QString const&, QStringList 
const&)
      at sources/qtbase/src/plugins/platforminputcontexts/compose/main.cpp:56
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QPlatformInputContext* qLoadPlugin1<>(QFactoryLoader const*, QString 
const&, QStringList const&)
      at 
../../include/QtCore/5.5.0/QtCore/private/../../../../../../src/corelib/plugin/qfactoryloader_p.h:107
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QPlatformInputContextFactory::create(QString const&)
      at sources/qtbase/src/gui/kernel/qplatforminputcontextfactory.cpp:65
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QPlatformInputContextFactory::create()
      at sources/qtbase/src/gui/kernel/qplatforminputcontextfactory.cpp:80
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QXcbIntegration::initialize()
      at sources/qtbase/src/plugins/platforms/xcb/qxcbintegration.cpp:260
      in /home/milian/projects/compiled/qt5/lib/libQt5XcbQpa.so.5
    QGuiApplicationPrivate::eventDispatcherReady()
      at sources/qtbase/src/gui/kernel/qguiapplication.cpp:1193
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QCoreApplication::init()
      at sources/qtbase/src/corelib/kernel/qcoreapplication.cpp:738
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QCoreApplication::QCoreApplication(QCoreApplicationPrivate&)
      at sources/qtbase/src/corelib/kernel/qcoreapplication.cpp:653
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QGuiApplication::QGuiApplication(QGuiApplicationPrivate&)
      at sources/qtbase/src/gui/kernel/qguiapplication.cpp:554
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QApplication::QApplication(int&, char**, int)
      at sources/qtbase/src/widgets/kernel/qapplication.cpp:563
      in /home/milian/projects/compiled/qt5/lib/libQt5Widgets.so.5

...

393.20KB consumed over 13 calls from:
QArrayData::allocate(unsigned long, unsigned long, unsigned long, QFlags<>)
  at sources/qtbase/src/corelib/tools/qarraydata.cpp:105
  in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QTypedArrayData<>::allocate(unsigned long, QFlags<>)
      at 
../../../../include/QtCore/../../../src/corelib/tools/qarraydata.h:217
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QVector<>::reallocData(int, int, QFlags<>)
      at ../../../../include/QtCore/../../../src/corelib/tools/qvector.h:505
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QVector<>::append(QComposeTableElement const&)
      at ../../../../include/QtCore/../../../src/corelib/tools/qvector.h:610
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::parseKeySequence(char*)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:440
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::parseComposeFile(QFile*)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:293
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::processFile(QString)
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:267
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::findComposeFile()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:112
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    TableGenerator::TableGenerator()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp:56
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QComposeInputContext::QComposeInputContext()
      at 
sources/qtbase/src/plugins/platforminputcontexts/compose/qcomposeplatforminputcontext.cpp:85
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QComposePlatformInputContextPlugin::create(QString const&, QStringList 
const&)
      at sources/qtbase/src/plugins/platforminputcontexts/compose/main.cpp:56
      in 
prefix/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
    QPlatformInputContext* qLoadPlugin1<>(QFactoryLoader const*, QString 
const&, QStringList const&)
      at 
../../include/QtCore/5.5.0/QtCore/private/../../../../../../src/corelib/plugin/qfactoryloader_p.h:107
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QPlatformInputContextFactory::create(QString const&)
      at sources/qtbase/src/gui/kernel/qplatforminputcontextfactory.cpp:65
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QPlatformInputContextFactory::create()
      at sources/qtbase/src/gui/kernel/qplatforminputcontextfactory.cpp:80
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QXcbIntegration::initialize()
      at sources/qtbase/src/plugins/platforms/xcb/qxcbintegration.cpp:260
      in /home/milian/projects/compiled/qt5/lib/libQt5XcbQpa.so.5
    QGuiApplicationPrivate::eventDispatcherReady()
      at sources/qtbase/src/gui/kernel/qguiapplication.cpp:1193
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QCoreApplication::init()
      at sources/qtbase/src/corelib/kernel/qcoreapplication.cpp:738
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QCoreApplication::QCoreApplication(QCoreApplicationPrivate&)
      at sources/qtbase/src/corelib/kernel/qcoreapplication.cpp:653
      in /home/milian/projects/compiled/qt5/lib/libQt5Core.so.5
    QGuiApplication::QGuiApplication(QGuiApplicationPrivate&)
      at sources/qtbase/src/gui/kernel/qguiapplication.cpp:554
      in /home/milian/projects/compiled/qt5/lib/libQt5Gui.so.5
    QApplication::QApplication(int&, char**, int)
      at sources/qtbase/src/widgets/kernel/qapplication.cpp:563
      in /home/milian/projects/compiled/qt5/lib/libQt5Widgets.so.5

[1]: http://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux

-- 
Milian Wolff | milian.wolff at kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel. Germany +49-30-521325470, Sweden (HQ) +46-563-540090
KDAB - Qt Experts - Platform-independent software solutions