[QBS] Building Qt with QBS

[Andrew Knight]

On 04/01/2015 05:58 PM, Oswald Buddenhagen wrote:
> On Tue, Mar 31, 2015 at 03:17:59AM +0300, Andrew Knight wrote:
>> For the past few months, I've been tinkering with using QBS to build
>> the Qt libs we use in our projects.
>>
> that was risky ... if i actually had the time to do what i wanted,
> you'd have duplicated quite some work. anyway, busy me, lucky you. ;)

Yes, lucky me. I was aware of the risk, but this was a good way to get 
my head around the requirements needed to move to an "evolved" build 
system and churn out packages at more reasonable rate.

>> The reason I'm posting is to just make it known that we (meaning my
>> colleagues and I) are actively working on this,
>>
> that's excellent news.
> what i'm interested in (possibly even more): how much effort do you guys
> want to spend on improving qbs itself? considering
> https://bugreports.qt.io/browse/QBS-70's longish list of dependencies
> required for a *clean* qbs-based qt build system, quite some work lies
> ahead.

I think we would be interested in helping out, especially if it improves 
life for us and/or our customers. We are invested in QBS already, after 
all. My crusade to git rm the .pro files from our repositories is 
largely complete... (*evil laugh*)

>> - The repository itself is just a collection of QBS files (and maybe a
>> few JS files as well as some pre-built artifacts I haven't bothered
>> writing rules for yet). That means it is designed to work with a
>> separate checkout (or even tarballs) of Qt sources (e.g. qt5.git). I
>> would also like to see it not tied to any particular Qt version, but to
>> support a range of Qt releases to allow quick switching between Qt
>> sources (read: branching of the repo is not tightly tied to the Qt version).
>>
> i understand your motivation for this, but from my perspective it's a
> total no-go - it's an utter maintenance nightmare.

The nice thing about QBS is that you can do a lot with globs and 
exclusions, so it's easier to do something like this in that sense. 
Obviously, you end up with the problem of checking that you don't break 
a different version of Qt when you make a change, so I agree that this 
is likely to be impractical in reality.

> i'd create throw-away wip/qbs-5.5 (and later wip/qbs-dev) branches in all
> relevant qt repos straight away. support for multiple versions can be
> achieved the usual way: merge from -5.5 to -dev.

Yes, I guess we can pretty much follow the qmake model here, with the 
top-level project living in qtbase and (generally speaking) a qbs file 
replacing each .pro/.pri. One thing I like about the centralized 
approach (i.e. putting the project(s) in qt5.git) is that the directory 
structure more easily take on a different shape. For example, qtbase's 
and qtwayland's platform project files could share the same hierarchy. 
You lose that with submodules. You also lose it if you are boring and 
follow the source hierarchy.

>> - There are two "prerequisite" steps to building Qt modules:
>
>> (1) build the host tools (using the host's profile)
>> Host tools can be reused for every target profile you decide to build
>> the framework for.
>>
> that's something i want to do independently from migrating to qbs.
> there are some challenges though, in particular how qmake ties it all
> together with its properties.

Yeah, by ignoring qmake, you basically can just build QtBootstrap and 
the host tools follow along willingly.

> the inverse conclusion is that this *should* not be a prerequisite for
> building with qbs, and i'd prefer to not have it unless we (you ;)
> solve the problem upstream before that.

Sure. Is there is already a way to create a QBS profile (using 
multiplexing) that lets you specify the compiler/sdk for host tools and 
a separate combo for the target platform? That's another problem I've 
chosen to ignore thus far, but Christian has suggested that it might be 
doable already.

>> (2) create a configuration file with all the various switches.
>> The configuration file (let's call it qtconfig.json) is basically a
>> portable configure command line.
>>
> well, that's what i wanted to do for the time being as well.
> in fact, i'd have been even cheaper than you: just use the existing
> configures and extract the relevant information from the generated .pri
> files. the reason for that is the thread you linked to yourself:
> whatever you hack up now will probably need a lot of rewriting when
> properly modularized configure support is implemented in qbs.
> i haven't thought much beyond qtbase, of course. :D

Yep, that's another option. Even so, configure is basically a long list 
of flags, most of them booleans, and some attempts to guess what you 
want. It seems like a really handy thing for allowing flipping a few 
bits and rebuilding only the parts of Qt which are affected, and 
something that QBS should be really good at. That's what started this 
exercise in the first place; to be able to disable some arbitrary Qt 
feature and (quickly) see what breaks.

>> It would at least be cool, to see this step also install the Qt module
>> configurations into a new QBS profile (perhaps as a replacement to
>> qbs-setup-qt, see [0]), as that might allow reusing more of QBS's
>> existing Qt module support.
>>
> not exactly ... that's something that should be created as a side effect
> of instantiating QtModule, etc. - https://bugreports.qt.io/browse/QBS-268

Yes, that's exactly the feature I was looking for :-) It would be nice 
to get to the point where "installing Qt" just means running 
qbs-setup-qt <magic> and pointing to your source tree. Then, Qt itself 
is built bit-by-bit based on your projects' dependencies...

>> qmake isn't even built with this setup, but a stand-in binary called
>> qhost pacifies the need for querying qmake variables.
>>
> well, that's obviously not going to fly for upstreaming, as qmake and
> everything around it is part of the qt api.
> but rectifying that as such is trivial. the challenges start with the
> decoupling of host and target qmake, as noted above.

I'm sure that bothers you more than it bothers me. I mean, requiring qbs 
already means requiring a pre-built host tool. A "self-aware" QBS could 
just try to build the host tools with the same toolchain it was built 
with, for example. That is, unless you're planning to do some fancy 
Canadian-cross Qt build...

Anyway, thanks for the feedback so far. I guess we could/should start 
with a qbs wip branch in one repo (qtbase, presumably) to get the party 
started. The other repositories can follow when there's a reasonable 
baseline established.

-ak