[Qt-creator] Bundling clang (for code model) with qt-creator package

Mon Dec 12 11:28:28 CET 2016

> On Dec 12, 2016, at 3:13 AM, Andrzej Telszewski <atelszewski at gmail.com> wrote:
> 
> On 31/08/16 09:12, Eike Ziller wrote:
>> The default path that the clang static analyzer checks is libexec/qtcreator/clang/ (+bin/clang etc)
>> http://code.qt.io/cgit/qt-creator/qt-creator.git/tree/src/plugins/clangstaticanalyzer/clangstaticanalyzersettings.cpp#n61
> 
>> Our clang build:
>> http://code.qt.io/cgit/qtsdk/qtsdk.git/tree/packaging-tools/build_clang.py
> 
> OK, I finally came to the point where QTC (4.2.0-rc1) does not build against my system LLVM (3.8.0).
> 
> I "parsed" qtsdk/packaging-tools/build_clang.py and qt-creator/scripts/deployqt.py for the clues on how to bundle Clang with QTC and got the idea. It actually seems to be quite simple... once you get the idea.
> 
> I detail my bundling steps below, together with some more questions.
> I actually tested this configuration and it seems to work fine.
> 
> *1) Configure, build and temporarily install LLVM:*

Btw you find the exact builds of LLVM that we use for our packages on
http://download.qt.io/development_releases/prebuilt/libclang/

> 
> $ tar xvf llvm-3.9.0.src.tar.xz
> $ mv llvm-3.9.0.src llvm-src
> $ cd llvm-src/tools
> $ tar xvf cfe-3.9.0.src.tar.xz
> $ mv cfe-3.9.0.src clang
> $ cd -
> $ cd llvm-src
> 
> $ mkdir build
> $ cd build
> $ cmake \
>    -DCMAKE_INSTALL_PREFIX=/tmp/llvm-ins \
>    -DCMAKE_BUILD_TYPE=Release \
>    -G "Unix Makefiles" \
>    -DLLVM_BUILD_TOOLS=OFF \
>    -DLLVM_INCLUDE_EXAMPLES=OFF \
>    -DLLVM_INCLUDE_TESTS=OFF \
>    -DLLVM_ENABLE_RTTI=ON \
>    -DLLVM_TARGETS_TO_BUILD="X86" \
>    ..
>  $ make
>  $ make install
>  $ # We disabled building of all the tools, but we still need some.
>  $ make llvm-config
>  $ cp -a bin/llvm-config /tmp/llvm-ins/bin
> $ cd -
> 
> Questions:
> 1. In build_clang.py, depending on "bitness" (which I guess is x86 vs x86_64), you set -DLLVM_TARGETS_TO_BUILD=X86 in the case of 32 bits and -DLLVM_TARGETS_TO_BUILD=AArch64 in the case of 64 bits. Why is it so? In particular, why AArch64?
> 2. As you can see, I passed some additional flags (not found in build_clang.py), like -DLLVM_BUILD_TOOLS=OFF, mainly to shorten the build time. Can you think of any troubles I might get with the cmake configuration as you see above?
> 3. Are there any important flags missing compared to the ones generated by build_clang.py?
> 4. Are there any other flags that would be worth adding? Please note that I removed -DCMAKE_C_FLAGS and -DCMAKE_CXX_FLAGS for clarity, I actually have them in my real build/packaging script.
> 
> When it comes to build_clang.py, this script expects some environment variables, like PKG_NODE_ROOT, CLANG_BRANCH, and more:
> 1. Where do they come from? How can I get them? :-)

They are defined as part of our build setup which is not publicly shared.

> 2. How can I determine which exact versions of software and any other flags you used to create the given QTC release?

It would probably be nice to define LLVM_REVISION and CLANG_REVISION within the Qt Creator repository. These will usually be the release sha1s from a llvm/clang release.

> 3. There is apply_patches(): how can I know if you applied some patches and if so, where can I find them?

That currently is not used. It was used for 4.1.0 to cherry-pick a single patch from llvm 3.9 to 3.8 which fixed an (fatal) issue with static analyzer and msvc.
Again it would probably be best to put any patches that we deem necessary for our Qt Creator release into the Qt Creator repository, though we’ll try to not need it in any case.

> *2) Configure, build and temporarily install QTC:*
> 
> $ tar xvf qt-creator-opensource-src-4.2.0-rc1.tar.xz
> $ cd qt-creator-opensource-src-4.2.0-rc1
> $ qmake-qt5 qtcreator.pro \
>    QTC_PREFIX=/usr \
>    IDE_LIBRARY_BASENAME=lib \
>    LLVM_INSTALL_DIR=/tmp/llvm-ins \
>    QBS_INSTALL_DIR=/usr
> $ make
> $ make install INSTALL_ROOT=/tmp/qtc-ins
> $ make docs
> $ make install_docs INSTALL_ROOT=/tmp/qtc-ins
> 
> $ # Install LLVM/Clang bits needed for running QTC.
> $ mkdir -p /tmp/qtc-ins/usr/libexec/qtcreator/clang/bin
> $ cp -a /tmp/llvm-ins/bin/clang /tmp/qtc-ins/usr/libexec/qtcreator/clang/bin
> $ cp -a /tmp/llvm-ins/bin/clang-3.9 /tmp/qtc-ins/usr/libexec/qtcreator/clang/bin
> $ mkdir -p /tmp/qtc-ins/usr/lib/qtcreator
> $ cp -a /tmp/llvm-ins/lib/libclang.so* /tmp/qtc-ins/usr/lib/qtcreator
> $ mkdir -p /tmp/qtc-ins/usr/libexec/qtcreator/clang/lib/clang
> $ cp -a /tmp/llvm-ins/lib/clang/* /tmp/qtc-ins/usr/libexec/qtcreator/clang/lib/clang
> 
> Questions:
> 1. In deployqt.py, you tweak rpath-s, but I found that simply copying libclang into /tmp/qtc-ins/usr/lib/qtcreator just works. So is this step necessary?

It was necessary once upon a time, I haven’t checked if it still is.

> As a side note, I'm only bundling clang, as all the other libraries (including Qt) are already installed system-wide.
> 
> *3. I create the package out of /tmp/qtc-ins and I'm done.*
> 
> General question:
> Have I missed something? ;-)

I don’t see anything obviously wrong ;)

> 
> Some final notes:
> - I'm building on x86_64 system,
> - both Clang Code Model and Clang Static Analyzer seem to work just fine with the described build/install instructions,
> - although I tried hard not to make a mistake, there might be some somewhere. I extracted and tweaked the commands from my build script for clarity. Anyways, the build and packaging works as expected using my build script.
> 
> I know I asked quite some count of questions.
> Thanks in advance for the help!
> 
> -- 
> Best regards,
> Andrzej Telszewski
> _______________________________________________
> Qt-creator mailing list
> Qt-creator at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/qt-creator

-- 
Eike Ziller
Principal Software Engineer

The Qt Company GmbH
Rudower Chaussee 13
D-12489 Berlin
eike.ziller at qt.io
http://qt.io
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B