[Qbs] Qbs and debian packaging
Wookey
wookey at wookware.org
Wed Dec 20 14:02:22 CET 2017
On 2017-12-20 11:09 +0100, Christian Kandeler wrote:
> On Wed, 20 Dec 2017 04:54:44 +0000
> Wookey <wookey at wookware.org> wrote:
>
> > I have a few things to package for Debian which use QBS.
> >
> > So I've had to work out how to make QBS fit into debian's build
> > concepts and packaging mechanisms. (e.g separate build and install steps)
> >
> > That has gone reasonably well (although some queries about best
> > practice remain), and I have
> > a) a package in debian (which so far as I can tell is the first one using QBS):
> > https://tracker.debian.org/pkg/dewalls
>
> I'm having trouble locating the actual project sources. Can you point me to them?
You can view it here:
https://sources.debian.org/src/dewalls/1.0.0+ds1-2/
or download the source here:
http://deb.debian.org/debian/pool/main/d/dewalls/dewalls_1.0.0+ds1-2.dsc
dget will pull all 3 files, or download the tarball and diff manually as well:
http://cdn-fastly.deb.debian.org/debian/pool/main/d/dewalls/dewalls_1.0.0+ds1.orig.tar.gz
http://cdn-fastly.deb.debian.org/debian/pool/main/d/dewalls/dewalls_1.0.0+ds1-2.debian.tar.xz
> Here are my comments on the wiki page.
> - Regarding the separation of build and install steps:
> 1) Even when not installing during the build, the install root should
> still be passed: qbs build --no-install modules.qbs.installRoot:<dir>.
Could you elaborate on why? rpaths is the only thing I can think of
that would be affected (if not actually installing) and we don't want
those pointing to temporary install directories.
> 2) I suggest passing the --no-build flag to the install command, for
> extra safety.
> - From (I think) qbs 1.9 on the profile no longer determines the name of the
> build directory. Instead, there is now the concept of a "configuration".
> Until (and including) qbs 1.10, this name is (unfortunately) given as
> a simple context-less parameter, like this:
> $ qbs build default // Same as "qbs build", dir is named "default"
> $ qbs build myConfig // dir is named "myConfig"
> From qbs 1.11 onwards, there will be a more sensible syntax:
> $ qbs build config:myConfig
> - Regarding the setup: As you have noticed, the current approach to
> storing/reading settings does not really consider anything but a
> "normal user". It'd be great if you could create a task at
> bugreports.qt.io and tell us more about your requirements.
Will do.
> - I find this sentence a bit misleading: "qbs is usually used with
> Qt although in principle it could work for other projects." It might
> be statistically true at this point, but the wording seems to imply
> that using it for non-Qt projects is purely a theoretical thing,
> which is really not the case. I'd prefer something like "qbs is often
> used for Qt projects, in which case the path to qmake [...]"
OK. I wasn't quite sure to what degree it is intended to be a
general-purpose build system, especially for languages other than
C/C++/ObjC
Thanks for reading the page and making comprehensive comments.
> - Regarding the lib paths: I suppose something along the lines of what
> you show there is the correct approach, though I would not necessarily
> expect project authors to provide debian-specific hooks, but rather
> generic ones. For instance:
> Project { // Or in some project-specific module
> property string libDir: "lib"
> // ...
> }
> DynamicLibrary {
> Group {
> fileTagsFilter: ["dynamiclibrary", "dynamiclibrary_symlink"]
> qbs.install: true
> qbs.installDir: project.libDir
> }
> }
> $ qbs build project.libDir:lib/$DEB_HOST_MULTIARCH
Good point.
> Note that "usr" should probably not be part of qbs.installDir, but
> rather is a possible value of qbs.installPrefix.
> - Environment variables can be read with Environment.getEnv().
> See https://doc.qt.io/qbs/jsextension-environment.html.
> - Setting cpp.cxxFlags on the command line should work. Note that your
> example has a typo (cxxflags with a lower-case f). If that's how
> you entered it, that would explain why it didn't work (though you
> should have gotten a warning).
>
> > It might be nice to have a debian support module in QBS to make stuff
> > 'just work'.
>
> https://bugreports.qt.io/browse/QBS-417
> You might want to add suggestions there.
>
> > Supporting multiarch
>
> I think I have addressed most of these above. If anything is still unclear, does not work or you simply disagree, please tell me.
>
> > There is also the issue for upstream to detect that this is a
> > 'debian-style' build and thus should use multiarch paths. There is
> > provision for detecting the OS, but not the distro. So perhaps it
> > should be enabled by some kind of flag? cpp.multiarchPaths perhaps?
> > Although I expect users would prefer it if the build system just DTRT
> > automagically. (Similarly debian almost never wants rpath set (except
> > plugins), and does want soVersions and dynamiclibray_symlinks so those
> > would be good defaults too)
>
> I would have thought that this sort of package-specific configuration was the job of the debian rules file. Am I wrong?
That's a somewhat philosophical question. Ultimately, yes, all such
details are for the packaging to get right, but it's good if the
upstream build does as much as reasonable 'right'. To what degree does
a user expect to get a 'bare upstream' build, or one built for the
OS/distro that they are running on? I'm not quite sure what the QBS
philosophy is here. It looks to me like it changes binary and library
paths automatically to suit the OS, so QBS knowing about multiarch and
using it when appropriate makes sense in this context. (Hopefully it
will become a sensible default on linux eventually, but currently only
debian-style distros use it by default. The main impediment to wider
adoption is the lack of a distro-independent mechanism for finding
canonical triplets for ABIs). On the other hand too much invisible
magic can be confusing and more explicitly specifying things makes it
clearer what is going on.
On rpaths QBS presumably needs to make heavy use for its 'local
install' mechanism to work, so wants to default them on.
> > Honouring buildflags
> > --------------------
> >
> > Debian has standard buildflags, e.g. for security settings, like
> > hardening flags, and default optimisation. It also has a mechanism for
> > enabling debug builds or disabling checks (e.g. for crossbuilding or
> > when they are slow), or disabling optimisation. See
> > https://www.debian.org/doc/debian-policy/#debian-rules-and-deb-build-options
> >
> > There are various ways to do this in the build process: typically
> > some env vars are set to be used in the build, but cammands can be run instead:
> > e.g.: CPPFLAGS:=$(shell dpkg-buildflags --get CPPFLAGS)
> >
> > https://wiki.debian.org/HardeningWalkthrough gives quite a lot of
> > examples for different build systems.
> >
> > I have not managed to make these work as I don't know if there is a
> > machanism to use env vars in qbs files. Is there? If not, how should
> > such info be passed in? Would we need to make equivalent qbs strings/lists?
>
> I have mentioned above how to get at environment variables, but unless we are talking about a specific debian module in qbs (see the linked task above), this looks like it's the wrong way around. Why not just pass the flags on the command line or put them in some sort of "packaging profile", e.g. via the qbs config command?
If the upstream .qbs file does not overwrite passed-in variables, then
yes just passing them in should work (and makes qbs behave like other
build systems). This was not working for me, (the dewalls .qbs file
sets cpp.cxxFlags and cpp.linkerFlags) which is why I asked about
precedence an appending rules. I'll experiment some more. (As you say -
maybe I just typoed the variable name.)
> > Also I haven't worked out the 'precedence rules' for QBS. i.e if
> > cpp.cxxFlags is set in the qbs file, and passed in via a qbs command,
> > does the latter take precedence? Is there a syntax for += (i.e. add
> > these options to the existing list) ?
>
> The precendence for module properties in decreasing order:
> - command line
> - project files
> - profiles
> - module prototype (i.e. the default values from the module file)
> Regarding the list semantics: Command-line overrides wipe out everything else (we might want to think about making that configurable), but in all other contexts lists are merged. In particular, profile contents simply replace the default values from the module prototype and thus get merged with what's in the project files. The latter is probably what you want.
Yes, that sounds promising.
> > qbs tried to avoid this sticky area of setting flags, because they are
> > toolchain specific, which makes sense, but we need a way of setting a specific set of things. The current dpkg-buildflags is:
> > CFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong -Wformat -Werror=format-security
>
> "-g" -> cpp.debugInformation:true
> "-O2" -> cpp.optimization:fast
> The rest would go into cpp.cFlags.
>
> > CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2
>
> "-D" -> cpp.defines
> The rest would go into cpp.commonCompilerFlags.
>
> > CXXFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong -Wformat -Werror=format-security
>
> cpp.cxxFlags for the "rest".
>
> > FCFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong
> > FFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong
>
> This is for Fortran? We have no fortran module yet.
>
> > GCJFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong
>
> Is gcj of any relevance these days?
>
> > LDFLAGS=-Wl,-z,relro
>
> cpp.linkerFlags, but without the "-Wl," escape.
>
> > OBJCFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong -Wformat -Werror=format-security
>
> cpp.objcFlags for the "rest".
>
> > OBJCXXFLAGS=-g -O2 -fdebug-prefix-map=/home/wookey/packages/cavewhere/dewalls/debian/dewalls-1.0.0=. -fstack-protector-strong -Wformat -Werror=format-security
>
> cpp.objcxxFlags for the "rest".
>
> > Does it make sense to add generic qbs names for all those things?
>
> No, I don't think so (in general; I have not evaluated all of the flags). We introduce convenience properties whenever there is a sensible abstraction that maps to different options on different compilers/linkers.
>
> > What happens when debian changes to a different set of options?
>
> Again, I was under the impression that it would be the packaging process' job to provide the correct set of flags to the build tool.
>
> > If not, how do we let qbs set what the package asks for (e.g. via
> > cpp.cLanguageVersion) whilst also honouring the required debian build
> > options?
>
> Not sure what you mean here. There should not be any conflicts.
>
> > Avoiding $HOME use
>
> Also commented on above.
>
> > Crossbuilding
> > -------------
> >
> > Does QBS support this, or is it intended to support this? I've not
> > tried yet. Setting the right toolchain commands: <triplet>-gcc,
> > <triplet>-binutils, <triplet>-strip, etc is a good start.
>
> There is nothing special about crossbuilding. Historically, the target architecture has been determined by the setup-toolchain command, but recently qbs has shifted more and more to being self-contained in this regard. Setting cpp.toolchainInstallPath, cpp.toolchainPrefix and qbs.sysroot should generally be enough.
I agree there is nothing very special about crossbuilding, but build
systems (especially new ones) regularly make assumptions that make it
difficult to have it working right :-)
I was wondering if the qbs-toolchain=setup --detect would interact
badly with crossing, for example, and how the packaging should
indicate the (build and) host architecture(s).
Thanks for the pointers. I'll test. In debian only cpp.toolchainPrefix
should be needed. Because of multiarch, there is no sysroot (or one
can consider it to be '/'). and crosstoolchains are installed in the
same paths as native toolchains. (although it remains possible to use
an external toolchain and sysroot if desired, which should all fit
quite nicely into this mechanism).
Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/qbs/attachments/20171220/141515fe/attachment.sig>
More information about the Qbs
mailing list