[Development] Qt 6 buildsystem support requirements

[Jason Newton]

On Mon, Jul 30, 2018 at 1:30 PM Matthew Woehlke
<mwoehlke.floss at gmail.com> wrote:
>
> On 2018-07-21 19:52, Jason Newton wrote:
> > I wanted to mention that this is on my mind alot for a few years days
> > as a user for a plethora of libraries.  My conclusion for the build
> > system with the brightest future is bazel [...]
>
> No. Just, *no*.
>
> Persistent build server? Java? No, thanks.

This is an option, you can keep it in a preference file, or you can
use it as a command line switch to not use this .   stat time
performance when monitoring many files, between invocations,
especially if globbing is involved and if you don't have a process
watching with inotify you will need to O(N) check every file for
modification and to hope your platform/hardware makes that check cheap
(windows doesn't,
https://github.com/mesonbuild/meson/blob/master/docs/markdown/FAQ.md#but-i-really-want-to-use-wildcards)
  Better to have a plan to deal with the case than not.  As for java
in the loop - this is a a build system, how much does it matter with
what it is written in if the implementation is good? I am not saying 0
but you should keep this number in check.  Make is made out of
something far more terrible than java.  Cmake is built on c++ but is
difficult to modify and the language they made is a mess.  Java works
in many situations and provides a write-once run anywhere mode that
works most of the time without issue and is performant enough for most
everyday tasks (better than python, and not always a long shot off
from all kinds of c/c++ depending on the application and exact
algorithms ), and importantly it works in this one without any notable
complaints.  OpenJDK has wide coverage for host-build environments:
http://openjdk.java.net - and is packaged by all major distributions,
first class.

>
> Maybe it's gotten better, but last I knew, bazel had no notion of
> "installing" software, which is just a non-starter. It's also heavily
> optimized for Google's internal usage, and (partly as a result)
> basically does not play at all nicely with software that's intended to
> exist in an open source, package-managed ecosystem.

It works well with open source actually, git is a first class data
source, mainly targeting external repositories, frequently github.
Source archive files are too.  The convention is that projects using
it embrace that and don't care about the most system packages if they
can get away with it.  Streamline switching out pkg_config based
results and package-kit or other kinds of datasources and really this
just works.  Discussing this problem upstream will probably get better
capabilities in the box well before expected shift time but your beef
is with closing the gap on package-managed dependencies outside of
bazel, and I'm all for closing it, although even now it is workable.


> (I'd be interested to know if there is *any* software packaged by any
> FLOSS distribution that uses bazel...)
>
> Basically, ask your distributors before even *considering* bazel. My
> suspicion is they're going to tell you that they refuse to package
> anything using it.
>
> And... seriously, *Java*?! Talk about bloat-ware... As dependencies for
> *a build tool* go, that's pretty insane. Especially if you're not
> planning to use it to build Java code.

As I said, ~300-400 megabytes for a JRE (~90MiB DL), in particular
openjdk , prospectively in a development environment where you've got
devel headers and packages installed.   The cost of the dependency is
neither bloat nor unbounded, it really is just that size on a machine
that's prepared to build software in 2018.  Please explain what issue
this presents.

>
> By comparison, CMake+ninja depend on... a C++ compiler. (Which is also
> needed for Qt itself, of course, so...) I haven't really looked, but I'd
> guess qmake is also pretty light-weight in the dependency department.
>
> > Why the qt project may be interested in this in short is:
> > -Fast and scaling bazel uses a long lived server technique that hangs
> > around
>
> Yeah... that's *just* the sort of thing we should be forcing on users...
> It's even *more* of an anti-feature if some other build systems needs to
> build Qt. (Yes, that happens.)
>
> On a related note, "hermetic builds" is pretty ironic. Your *build*
> might be hermetic, but bazel itself is *far* from... it's very reliant
> on putting all its garbage in "magic locations" in your home directory,
> unlike most build tools that only need to write to your build directory.
> Again, *not* friendly for anyone that needs to build Qt as an external
> dependency.

Actually that's just a default location - the output location you're
talking about is the outputRoot/output_base setting:
https://docs.bazel.build/versions/master/output_directories.html
Again, like gitignore or clang-format, a project can specify it's own
settings and the point of contention is default values with
persecution on capability. I think there's open path ways for change
behaviors that should be changed if there's lingering concern over
features like this that deal with use cases and problems rather than a
build system which does less and has less to worry about, notably the
cost of a build system that's too simple is a build system that's too
limit.  One of the things I like about the way bazel keeps this is I
can jump back and forth between an optimized build and a full debug
build (for example) - they don't erase eachother and you can only
achieve that jumping capability, being cheap, if you add a file-level
layer of indirection. The magic locations actually aren't magic at all
and are fully deterministic, hashes like git commits essentially with
1 level of symlinks sitting in the users build directory.  You could
easily have all your outputs live adjacent to (at minimum) to your
project/src directory, somewhat like the build convention in
cmake/autotools.

>
> > The amount of time that useful work is being performed is  greater
> > than I've observed in large cmake projects, even using ninja (assuming
> > ninja would work across projects that size, due to frequent make
> > sensitivities)
>
> Um... very few projects don't work with Ninja, and I've seen plenty of
> very large projects that *do*.

Yes, this has changed with time but sensitives can exist, I'd still
say it's a crap shoot when pulling down sources.

>
> > -Ability to build external libraries from source
>
> Yeah... no. Building third party libraries yourself is *evil*. And a
> great way to piss off package maintainers. No open source project should
> *EVER* rely on, or preferably even use by default, this ability.
>

It is not evil in any way.  It's a different approach that works well
for different concerns to attempt to make a ~constant cost vs a linear
cost in os * distribution * version.  What is needed is a way to do
both and make both sides of this happy.    Ways to easily lean on a
system install or if you're a dev on a system that doesn't play ball
well (version, timing, whatever), provide gradieted ways to use source
build overlays without tremendous headaches - for getting the latest
stable without going ppa for instance and working in custom build
flags for any sources of interest.  Bazel doesn't deal with system
packages in a first class way - there's no direct pkg_config
integration, though there are simple trivial scripts that use it that
are third party that should move to the builtins.  I can help make
those kinds of changes, if need be.  However I've had no problems
pulling in system libs (pkg-config or otherwise) and most of the time
packing up a library is smooth sailing, when building from source.  As
I said you get both and you don't have to pay the penalty of out of
band source builds just because by convention people not on a
distro-packagers side do.  And it's nice when you're working on a
software application and want to be distribution agnostic, as is
frequently the case if you are not a packager for a distribution.

Again pointing out sources-JIT package services like vcpkg, buckaroo,
and conan exist and have been gaining traction (users and word of
mouth).  The want for these capabilities in c++ is growing and bazel
provides a streamlined mechanism for it, but it is not generally
coupled to the internet for it, and allows gradiantions/mixes of that
JIT-from-sources and the system libraries.  On JIT from sources, that
gives you better support for esoteric platforms that receive little to
no TLC from package maintainers, for instance on why that has value
(S390 or arm distros are good examples)

Also wanted to mention a few things I forgot previously:

-Analysis tools for the dependencies, like graphviz, but also memory
usage and timing info.  A legitimate profiler, have a look:
https://docs.bazel.build/versions/master/skylark/performance.html
https://www.kchodorow.com/blog/2015/09/18/build-y-u-go-slow/
(admittedly few online examples of this output and this is looks
different now IIRC)
-Does a good job of controlling verbosity while keeping the user
informed on the running targets being processed concurrently while
giving a quick way to hop out, go serial or show commands executed
-Workable, like cmake workable, visual studio generator, and xcode outputs.
-clion has a plugin good direct plugin for it, though more will follow
and they tried to make it easy for IDEs to incorporate (
https://blog.bazel.build/2016/06/10/ide-support.html we can agree it'd
be create if more build systems did this, right?
https://docs.bazel.build/versions/master/ide.html for fuller list )

-Jason