[Qbs] Interfacing Qbs to Conan

[Richard Weickelt]

Jochen,

> Basically, we are doing it the way you described. However, we decided not
> to make a conan module which the generated modules depend on because we
> expect that this would lead to multiple invokations of `conan install`
> (one for each generated module) which we want to avoid. 

This should not be the case. Probes are cached by the location of their
definition and the input properties (snapshot before running configure).

> Yeah. While working on the Conan ModuleProvider, we encountered a few
> obstactles. Luckily, it seems we are able to work around them. I list
> these obstacles here along with the workarounds we are currently
> implementing so we can discuss them. I would then create a feature
> request ticket to extend the ModuleProvider accordingly.

It would be great if Qbs 1.16 could offer a complete Conan story. We have
roughly a month until feature freeze.

> So the following things might be helpful when writing a ModuleProvider:
> 
> 1.) Access to (project-level) probes It is quite likely that a
> ModuleProvider needs some external tools or files and those should be
> detected using probes. In our case, we need to detect the conan
> executable and the conanfile(s). Our workaround is to let the
> project-level probe execute `conan install` so the ModuleProvider does
> not need to know the location of the conan executable and the
> conanfile(s). This then produces a conanbuildinfo.json which the probe
> places in a defined location inside the build directory so that the
> ModuleProvider is able to find it.

It might be beneficial if module providers:

a. could define their own probe items.
b. have access to the project and the product they live in.
c. could be set up in projects like it is possible in products

I do not think they should have direct access to probes/properties defined
in the project/product by the user because this would be an "invisible"
dependency. That means: A provider should not rely on project.mycustomProperty.

> 2.) Access to the project the ModuleProvider is running in A
> ModuleProvider might need or save some files in the buildDirectory. So
> access to the project.buildDirectory and project.sourceDirectory would be
> helpful. In our case, we need to locate the conanbuildinfo.json in the
> buildDirectory. Enhancement 1.) might also solve this. However, in our
> project, we might end up with several conanbuildinfo.json files
> (hierarchy of projects) and therefore, access to project.name to
> differentiate the files might be helpful. Our workaround to get the
> buildDirectory is to assume that the outputBaseDir of the ModuleProvider
> is inside the build directory. So we simply do something like `var
> buildDirectory = FileInfo.cleanPath( outputBaseDir + '/../..' )`. For the
> multiple conanbuildinfo.json files issue, we simply take all of them
> inside a defined directory.

You can define custom properties for your module provider.

Are you aware that module providers operate on product level? You can
(currently) configure them in the profile and in the product. Would the
following approach do what you want?

// module-providers/conan/provider.qbs
// Sets up a module for each key in dependencies
ModuleProvider {
    property var dependencies
    // ...
}

Project {

    property var dependencies: conan.dependencies

    // https://codereview.qt-project.org/c/qbs/qbs/+/288927
    Probes.ConanfileProbe {
        id: conan
        conanfilePath: "conanfile.py"
    }

    Product {
        Depends { name: "conan.somelib" }
        moduleProviders.conan.dependencies: project.dependencies
    }
}

> 3.) Access to the full name of the Depends items for which the
> ModuleProvider was called This works for the fallback ModuleProvider
> AFAIK. However, at the moment there can be only one fallback
> ModuleProvider AFAIK. So it would not be easily possible for a project to
> use both pkgConfig and Conan, right? Allowing access to the full name of
> the Depends item in a regular ModuelProvider when evaluating the
> relativeSearchPaths property would allow writing something like `Depends
> { name: "conan.somePackage" }` which would invoke the conan
> ModuleProvider which would then generate the Module for "somePackage". 
> Our workaround is to generate Modules for all dependencies from the
> conanfile, no matter if they are used as Depends items in Qbs or not. So
> this is not a real issue in our case but it limits ModuleProviders which
> do not have something like the conanfile but only rely on the name of the
> dependency.

True. There should be a way to select fallback providers. At least you can
override the fallback provider by putting your own provider into a
somePath/module-providers/__fallback folder and add somePath to qbsSearchPaths.

I wonder how a better and generic solution would look like. We could add a
moduleProviders.defaultFallbackProvider configuration option or we could
replace Depends.enableFallback by Depends.providers and then explicitly
specify a provider.

We could also add the capability to "configure" the fallback provider via
the Depends item so that we could write:

Depends {
    name: "mylib"
    fallbackProviders: [
        "conan",
        "pkgConfig"
    ]
}

Other thoughts?

> 4.) Mechanism to force execution of the ModuleProvider Since
> ModuleProvider generate modules on a dynamic basis, it is not unlikely
> that the environment changes and the ModuleProvider needs to re-run. At
> the moment, this can be forced with the --force-probe-execution command
> line parameter. However, it would be nice if Qbs could detects this
> automatically. I'm not sure how this could be realized. Maybe a property
> on the ModuleProvider that, when it evaluates to `true`, forces
> re-execution of the ModuleProvider? In our case, we would like to re-run
> the ModuleProvider when the conanfile (or conanbuildinfo.json) has
> changed. Our workaround is that we have to call qbs with
> --force-probe-execution manually when the conanfile has changed.

https://codereview.qt-project.org/c/qbs/qbs/+/288927 achieves automatic
probe re-execution by assigning the modification time stamp of the conanfile
to a property of the probe.

An --force-module-generator-execution command line flag might be worth to
add to qbs resolve.

Richard