[Qbs] tst_blackbox-qt fails

[Richard Weickelt]

Christian,

>> I tried to build Qbs from source and to run the autotests on Debian stretch
>> with Qt 5.9.7 and 5.12.0. After creating a Qt profile (with the built Qbs)
>> and setting the undocumented QBS_AUTOTEST_PROFILE variable I still see some
>> tests failing in tst_blackbox-qt (see log below).
>
> I strongly suspect that you did not run the autotests with the exact same
build that you called setup-qt with. Please double-check.

Thanks, that was it. I wasn't aware that Qbs stores profiles in different
folders for each Qbs version. I assumed all Qbs versions share a single
folder with profiles.

This is how I build and test now:

qbs build profile:qt
qbs run -p qbs_app profile:qt -- setup-toolchains --detect
qbs run -p qbs_app profile:qt -- setup-qt
/usr/local/Qt/5.9.7/gcc_64/bin/qmake qt
QBS_AUTOTEST_PROFILE=qt qbs build -p autotest-runner profile:qt

Is this how it is supposed to be done?

>> I need some guidance how to properly setup the autotests. In addition, I
>> would like to know how The Qt Company is running Qbs regression tests today
>> on which platforms and when.
>
> This is currently done on a local Jenkins-based CI. Tests run on Linux (32
and 64 bit), macOS (different versions) and Windows. On Windows, MSVC and
mingw are tested. For the Qt autotests, we also test against a static Qt
installation on Windows.

Is this visible outside your company?
How could external contributors leverage that infrastructure? E.g. how could
they invoke tests and see the results?
Would it be possible to integrate that into gerrit? And if yes, would you do it?

>> The page at
>> http://doc.qt.io/qbs/building-qbs.html does not talk about autotests and the
>> docker images are outdated. Thus I assume that the information on this page
>> is not used.
>
> I don't know what you mean by that.

1. The page is titled "Building Qbs" but doesn't explain how to run
autotests and the requirements on the environment. If it was explained I
wouldn't have asked above.

2. It is not possible to build and test the Qbs master branch with the
provided Docker images without issues. They come with Qt 5.9.3 and Qbs 1.9.1.

2.1 Building with Qbs fails:

docker pull qbsbuild/qbsdev:stretch
docker run -it -v $PWD:/qbs -w /qbs qbsbuild/qbsdev:stretch

# In the docker shell
qbs setup-toolchains --detect
qbs setup-qt /usr/local/Qt/5.9.3/gcc_64/bin/qmake qt
qbs build profile:qt

No build graph exists yet for this configuration.
Resolving project for configuration default
ERROR: /qbs/tests/auto/blackbox/blackbox-joblimits.qbs:3:1 Errors in product
'tst_blackbox-joblimits':
/qbs/qbs-resources/modules/qbsbuildconfig/qbsbuildconfig.qbs:6 Can't find
variable: project
/qbs/src/lib/qtprofilesetup/qtprofilesetup.qbs:3:1 Errors
[...]

2.2 Building with qmake fails as well:

/usr/local/Qt/5.9.3/gcc_64/bin/qmake -r qbs.pro && make
[...]
g++ -Wl,-z,origin '-Wl,-rpath,$ORIGIN/../lib' -Wl,-O1
-Wl,-rpath,/usr/local/Qt/5.9.3/gcc_64/lib -o ../../../bin/qbs-config-ui
consolelogger.o coloredoutput.o commandlineparser.o main.o mainwindow.o
moc_mainwindow.o   -L/qbs/src/app/config-ui/../../../lib -lqbscore
-L/usr/local/Qt/5.9.3/gcc_64/lib -lQt5Widgets -lQt5Gui -lQt5Core -lGL -lpthread
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
Makefile:246: recipe for target '../../../bin/qbs-config-ui' failed
make[2]: *** [../../../bin/qbs-config-ui] Error 1

These things are easy to fix, but I assume that nobody has tested the
instructions on this page for a while and nobody is using those Docker
images. I am going to make a patch.

Richard