[Qbs] Qbs (way) faster than cmake (or benchmark issue)

[Christian Gagneraud]

On Mon, 22 Jul 2019 at 22:10, Oswald Buddenhagen
<oswald.buddenhagen at gmx.de> wrote:
>
> On Mon, Jul 22, 2019 at 11:00:07AM +1200, Christian Gagneraud wrote:
> > At the 20 jobs mark, cmake start to stagnate, whereas qbs still make
> > use of parallelism, at the 30 jobs mark, cmake completely stopped
> > reducing overall build time, whereas qbs start to stagnate. At the 40
> > jobs mark, both systems are stale.
> >
> that means that qbs creates a build graph with fewer bottlenecks. cmake
> certainly could do that with relatively little effort as well, at the
> likely cost of slightly increased null rebuild times (due to the graph
> being bigger).

Should i plot qbs vs cmake null rebuild times? That would be funny if
qbs was faster again... :P

Just kidding...

I think the build graph generated by cmake is too big/complex.
By default, Qbs doesn't track dependencies on system header files for example.

We shouldn't see such a big difference b/w cmake/ninja and qbs.
Although i like Qbs very much, i do not believe that it implements
revolutionary algorithms compared to ninja.
Ninja website is actually very interesting (eg. w/ references to
http://gittup.org/tup/ and https://github.com/apenwarr/redo) and my
gut feeling is that this tool is really efficient in executing it's
input.

I think (on of) the problem is what cmake generates, as mentioned in
an earlier email, I find suspicious that ninja has to call cmake to
handle moc. And this moc aggregation is certainly one of the
bottleneck (again, just gut feelings).

> some suggestions for making a diagram where the interesting stuff isn't
> crammed into a narrow strip at the bottom:
> - a logarithmic job count axis

Doesn't really help on it's own, but log on both axes kind of do, see
attached document.

> - build speed per job vs. job count
> - cpu utilization per core vs. job count

The main issue here is that i've used '-j' to control jobs, which is
not good, i should have used docker --cpus instead.
Nonetheless all the data is attached if someone fancy some data mining.

> - <your idea here>

I don't think i'll spend more time on this. But feel free to play with
that if you (or anyone else) want.
https://gitlab.com/chgans/cubicninjas

Nothing fancy really, it's just a Dockerfile and a build script.

I've attached the csv files, there's lot of potentially interesting
data in build-qtcreator-stats.csv (see
https://linux.die.net/man/1/time, i've dumped everything)
The other file (build-qtcreator-dstat.csv) is data collected by dstat
for the whole duration of the builds (see
https://linux.die.net/man/1/dstat, i've dumped disk, load and cpu
data).
Again, it's full of potential useful data, it just needs some mining...

Docker command:
cd /path/to/qt-creator
git clone https://gitlab.com/chgans/cubicninjas.git
docker build -t cubicninjas cubicninjas
mkdir builds
docker run -it -v $PWD:$PWD -w $PWD --mount
type=tmpfs,destination=/tmp --mount type=tmpfs,destination=$PWD/builds
cubicninjas $PWD/cubicninjas/build.sh

basic gnuplot command for build-qtcreator-stats.csv:
set datafile separator ","
set logscale x
set logscale y
plot 'build-qtcreator-stats.csv' every 2::1 using 2:19 with
linespoints title 'ninja', '' every 2::0 using 2:19 with linespoints
title 'qbs'

Enjoy and my apologies for the grumpy email yesterday.

Chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build-qtcreator-stats.csv
Type: text/csv
Size: 10850 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/qbs/attachments/20190723/b5ca7ccf/attachment-0001.csv>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: qtcreator-build-time-cmake-vs-qbs_loglog.pdf
Type: application/pdf
Size: 10106 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/qbs/attachments/20190723/b5ca7ccf/attachment-0001.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build-qtcreator-dstat.csv.gz
Type: application/gzip
Size: 323084 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/qbs/attachments/20190723/b5ca7ccf/attachment-0001.gz>