[Qbs] tst_blackbox-qt fails

[Christian Kandeler]

On Wed, 2 Jan 2019 17:27:29 +0100
Richard Weickelt <richard at weickelt.de> wrote:

> This is how I build and test now:
> 
> qbs build profile:qt
> qbs run -p qbs_app profile:qt -- setup-toolchains --detect
> qbs run -p qbs_app profile:qt -- setup-qt
> /usr/local/Qt/5.9.7/gcc_64/bin/qmake qt
> QBS_AUTOTEST_PROFILE=qt qbs build -p autotest-runner profile:qt
> 
> Is this how it is supposed to be done?

Looks fine to me. For extra safety, e.g. if simultaneous execution with other qbs instances is required, one would also pass a dedicated --settings-path, as to have truly unique profiles for the respective run.

> >> I need some guidance how to properly setup the autotests. In addition, I
> >> would like to know how The Qt Company is running Qbs regression tests today
> >> on which platforms and when.
> >
> > This is currently done on a local Jenkins-based CI. Tests run on Linux (32
> and 64 bit), macOS (different versions) and Windows. On Windows, MSVC and
> mingw are tested. For the Qt autotests, we also test against a static Qt
> installation on Windows.
> 
> Is this visible outside your company?
> How could external contributors leverage that infrastructure? E.g. how could
> they invoke tests and see the results?

Not at all.

> Would it be possible to integrate that into gerrit? And if yes, would you do it?

I suppose it would be possible to put qbs under the CI regime just like Qt. Jake has actually tried that at some point, but ran into problems. I don't remember the details. I have a suspicion that this is one of those tedious tasks that everybody likes to leave to someone else...

> >> The page at
> >> http://doc.qt.io/qbs/building-qbs.html does not talk about autotests and the
> >> docker images are outdated. Thus I assume that the information on this page
> >> is not used.
> >
> > I don't know what you mean by that.
> 
> 1. The page is titled "Building Qbs" but doesn't explain how to run
> autotests and the requirements on the environment. If it was explained I
> wouldn't have asked above.

That's because it's aimed at users of qbs who want to build from source, not developers of qbs.

> 2. It is not possible to build and test the Qbs master branch with the
> provided Docker images without issues. They come with Qt 5.9.3 and Qbs 1.9.1.

I have close to zero knowledge about the Docker images, unfortunately. So anyone investigating in this area for a couple of minutes will already likely have a better understanding of it than I do.

> These things are easy to fix, but I assume that nobody has tested the
> instructions on this page for a while and nobody is using those Docker
> images. 

Sounds plausible.

> I am going to make a patch.

Thanks.


Christian