[Development] QtCS2019 Notes: Clang-based cpp parser for lupdate

Joerg Bornemann Joerg.Bornemann at qt.io
Thu Nov 21 17:38:28 CET 2019

You can find the session notes at 

...and here in plain text:

== Clang-based cpp parser for lupdate - Session Notes ==

The old C++ parser in lupdate has several problems, for example
* It's lacking support for Objective C/C++ (QTBUG-44808)
* It lacks support for a lot of modern C++ constructs. "Modern" is here 
C++11. See QTBUG-42744, QTBUG-53850 and QTBUG-63373.
* It is deemed hard to maintain.

There is the ongoing effort to replace the old outdated parser with a
clang, similar to what's been done in qdoc. The corresponding JIRA
issue is QTBUG-71835.

Work is done in dev and therefore targeting Qt 6.

QDoc uses clang's C API, whereas lupdate uses clang's C++ API, which
means library requirements and thus configuration are a bit more
involved than qdoc's.

The old parser will still be available as fallback if build
requirements are not met. The parser can also be switched on run time.

The new clang-based parser needs all compiler flags to "fake build"
the project. A compilation database can be provided. Concerns were
raised where this compilation db is supposed to come from. Luckily,
CMake is able to create one.

Current Status:
It works and produces seemingly the correct output, able to consume 
modern C++ constructs.
But it's sloooooow. Running on Qt Creator takes 1 min with the old 
parser, and 50 min with the new one.

Proposed Solutions for the speed problems:
* precompiled headers are under investigation
* jumbo builds (include all sources from a central file)
* running the AST creation in parallel
* restrict the AST even more, e.g. tell clang to leave out information 
we don't need

Due to the usage of clang the source that's scanned for translations 
must be buildable with the provided options. That implies:
* lupdate does not see #ifdef'ed code that's not compiled for the 
current platform
* lupdate does not see source files that are not built for the current 
build configuration

How to overcome this?
Unless we can convince clang to be more lenient towards non-buildable 
code, we could instead
* Use the more lenient parser of clang-format?
* Use Qt Creator's (non-clang) C++ parser instead of the lupdate one?
* Rumors have it that work on clangd is done in that direction.

More information about the Development mailing list