[Qbs] Interfacing Qbs to Conan

[Richard Weickelt]

Hello Leon,

I am really glad to read that Clausmark is looking into interfacing Qbs with
Conan. Since Conan gets more and more traction and brings great advantages,
Qbs should support it well. That's why I want to share some more detailed
thoughts on this topic. I'll start with what I originally had in mind and
later get to module providers.

> Here's the probe. I is instantiated at project level where the value of
> 'generatedQbsFilePath' is added to the 'references' property. If you run
> it like this, 'targetArch' will be undefined.

I assume that this has already been sufficiently answered and since I was
told that you are now trying to implemented a conan module provider, I will
not comment on it. However, I had this topic on my personal backlog for a
while and came up with the following probe (WIP):
https://codereview.qt-project.org/c/qbs/qbs/+/288927

You may find this useful and I would be glad to get feedback.

> How would I access a property set in a profile? I think I've tried it
> before but I probably did something wrong.

Module properties set in profiles are only accessible from within products
and modules.

> Also in the blog post about the release of qbs 1.15.0 it says the
> following: Probes on the project level are now evaluated before Profile
> items. That way a Qbs project can easily interface to package managers
> like Conan or vcpkg and resolve all build dependencies including compiler
> toolchains in a clean and platform-agnostic way. [...]

As Christian already wrote, this is about Profile items. My assumption is
the following:

1. When using Conan, you want to package up _everything_ necessary to build
your project. Not only the relevant libaries, but also toolchains. You want
to have the same tools on every computer as well as in your CI system and
you might have different host operating systems.

2. You have modules for all relevant libraries and your only problem is the
connection from your conanfile to setting up these modules.

3. You do not want to or cannot use conan's qbs generator because your
packages are missing important meta information or do not fit into conan's
standard layout. Conan't built-in qbs generator is very inflexible.

Thus, I thought users could write a simple Probe to run conan install and
extract all relevant meta information from the resulting
conanbuildinfo.json. Since Probe items are now executed before Profile items
are evaluated, users could do something like:

Project {
    Probes.ConanfileProbe {
        id: conan
        conanfilePath: path + "/conanfile.py"
        options: ({someOpt: "bla"})
    }

    Profile {
        name: "gcc"
        mylib.includePaths: conan.dependencies["mylib"].include_paths
        qbs.toolchain: "gcc"
        cpp.installPath: conan.dependencies["gcc-arm-none-eabi"].rootpath +
                         "/bin"
    }
}

This way you could set up a completely self-containing project and you would
never again mess around with manual dependency installation on your computer
(toolchains included). Note how simple, straight-forward and
easy-to-understand this is compared to how CMake interacts with Conan.

However, there is unfortunately a limitation: Qt Creator is not able to use
profiles with a custom name specified in a project. Qt Creator can only
invoke builds with its own cryptic profile naming scheme (for instance
qtc_Desktop__9d3c5e00). One possible, but ugly work-around for this problem
is to create a dummy profile that shadows an existing profile in Qt Creator.
Like this:

    Profile {
        name: "qtc_Desktop__9d3c5e00"
        baseProfile: "gcc"
    }

Qbs will then no longer use the values from Qt Creator's own profiles, but
the ones set in the Profile item in the project. That means, developer
participating in the project would need to add such a dummy Profile item to
the project which is not really what we want.

I think it would be a great improvement in QtCreator to extract Qbs' Profile
items as read-only kits when opening a Qbs project. And it would be multiple
times easier than anything existing today.

> Doesn't that mean that profile specific properties are inaccessible at
> the time the probe is evaluated?

True (on project-level). The "qbs" module is a bit special because it is
loaded very early and is always implicitly available everywhere. The
qbs.architecture property is another oddity because if it is not set by the
profile, it will be set later by the cpp module.

> How would I reference the qbs file generated by conan though? I'm only
> doing it with this project level probe because I thought there was no
> other way, or is there?

If you want to reference the qbs file created by conan's qbs generator, then
yes, would need a probe on Project level invoking conan install
/path/to/conanfile -g qbs ... The qbs file should be generated somewhere
under project.buildDirectory.

Back to the topic of module providers, I do think that your idea to write a
generic module provider for Conan is a great idea and would be a perfect
companion for a ConanfileProbe when dealing with library packages that
follow Conan's default metadata layout properly.

There might be different solutions to this, but what I think would be simple
and straight forward:

1. Make the conan provider a fallback provider and let it catch any module
request where no built-in module exists. The conan provider does not need to
know anything about conan, the conanfile or whatever. It is possible though
to configure module providers on product level. See
https://code.qt.io/cgit/qbs/qbs.git/tree/tests/auto/blackbox/testdata/module-providers/module-providers.qbs#n6.

2. Create a very simple module template like this:
    Module {
        Depends { name: "cpp" }
        Depends { name: "conan" }
        cpp.includePaths: conan.dependencies[$PACKAGE_NAME$].inclue_paths
        ...
    }

3. Implement a conan module that does all the dirty work:

    Module {
        property string installPath // install path of Conan
        property string conanfilePath
        property var options
        property var settings
        property var dependencies: conanProbe.dependencies

        Probes.ConanfileProbe {
            id: conanProbe
            executable: installPath ? installPath + "/conan" : "conan"
            conanfilePath: conanfilePath
            options: options
            settings: settings
        }
    }

This conan module can now be easily configured in profiles or products. You
don't need to take the route via the separate conan module. Instead, you
could just put the ConanfileProbe into the module template. But I believe
that the extra conan module would be a bit cleaner.

I hope this gives you some ideas how to move forward and I would be more
than happy to discuss any issues appearing on the way.

Best regards
Richard