[Qt-creator] importing "foreign" cmake projects remains problematic

[René J.V. Bertin]

On Thursday June 08 2017 21:00:28 Tobias Hunger wrote:

Hi Tobias,

>I probably do not fully understand what you are doing, so please be
>patient with me:-)

So for the sake of clarity let me just show a few commands from a typical workflow. I already indicated that the context is MacPorts packages, its so-called ports which are manipulated using a command that's (confusingly?) called port itself. When developing them, or even when working on a software project bundled as a port (there are some advantages to that), I typically do

Fetch code if not done yet, unpack, apply any local patches (port fetch, port extract, port patch) and run the configure step (showing progress):
%> port -v configure foo [+build_variants] [configure.optflags=...] [configure.compiler=...]
Run the build step:
%> port build foo [+build_variants]
Install to a local "destroot" location
%> port destroot foo [+build_variants]

Those could be done in a single invocation too: `port destroot  foo [+build_variants] [configure.optflags=...] [configure.compiler=...]`.

The steps that interest me here are the 1st two, especially the configure step itself. This is done in a special limited environment, possibly tailored for the project and with arguments to the configuration utility (e.g. cmake) that ensure a reproducable build that is exactly as it's intended to be. There's usually not a lot of need to generate very long/complex autoconf/configure command lines, contrary to cmake which tries to be too clever about too many things.

It may seem contrived to want to use an IDE in combination with this kind of workflow and it's undoubtedly overkill when you just have to maintain a few "distribution patches". After all, those port commands can be seen as the commandline equivalent of their equivalent IDE features. But you can also see this as a "meta build system" that helps you formalise the whole process, making it trivial to reproduce the exact same set-up elsewhere or from scratch. And then there's of course the fact that it also allows to install and uninstall your development code much more cleanly (and in MacPorts' case, simply activate a previous, still installed version when desired).

>Thank you for taking the time to provide your feedback instead of just
>running off to greener pastures!

My pleasure! And you know, pastures always look greener from the other side of the fence so I always keep an eye out on alternatives (though more alertly so when the grass under my feet seems to be going brown or getting thin ;))

>Can you register your wrapper script with creator? That should work,
>provided it it (mostly) transparent and forwards the relevant
>arguments to the real cmake.

That's what I have been trying: https://github.com/RJVB/macstrop/blob/master/kf5/kdevelop5/files/cmake2port

But it's important to realise that this is a very indirect wrapper to cmake, without any possibility to hand off arguments to the cmake invocation that counts.

This approach almost works, and protects the build directory from unwanted or unexpected changes an inappropriate cmake command might make. But I've seen a case yesterday where all of a sudden my CMakeCache was gone, as if deleted by Creator. That's annoying because MacPorts maintains its own statefile: once a build phase like `port configure` completed successfully it is marked as done and will be skipped the next time.
Then there's the fact I haven't been able to get the actual sources to show up in the project, even if it built as expected.

It's a bit complicated though to get this right with Creator's concept of build kits. And I get the impression that Creator also doesn't know how to obtain a project's targets from the cmake information, and then allow building the selected target from the project tree view.

>> What's really missing from Creator is support for a separate configuration step with exact control of the arguments to be used.
>
>What do you mean here? You want the call to cmake to configure the
>project to be more explicit?

Basically, yes, I miss a possibility to set up the configure step in the project configuration page. I presume you get to do this when you create a new project, but even for such projects, how do you repeat that step with the appropriate arguments?

>Make/ninja are calling out to cmake all
>the time. Why would creator not do the same? It needs the same level
>of information.

2 reasons: make/ninja already to it (so why would Creator do it "all the time" too?). And make/ninja will raise an error in situations where running cmake (without all the appropriate arguments) will just wipe the error silently and set up things differently.

Just as an example, something I'd consider a bug in cmake. CMake will get compiler options from the environment (CFLAGS & CXXFLAGS) and add them to the CMAKE_C*_FLAGS variable corresponding to your current CMAKE_BUILD_TYPE. If however you rerun cmake with a different compiler selection it will do a sort of partial reset halfway through its run, and at that point it will ignore those compiler flags. IOW, the only way I found till now to reconfigure for building with a different compiler AND use the desired compiler options is to do a `rm -rf $build.dir/CMake*` before running cmake with all necessary arguments.

>It starts cmake and asks it for information on the project you opened
>directly. At least it does in server-mode. With the older

I already had a run-in with CMake's server mode in Kdevelop, which made me go back to a branch that doesn't use that mode. IIRC there are issues with it that won't be resolved until 3.9 .

But even if it *has* to do this instead of just using the current version of the CodeBlocks project (or compile_commands.json) file maintained by cmake, it would be safer if cmake were called with all its original arguments.

FWIW, KDevelop also used NOT to do that, but now does and also provides an interface to check and modify all aspects of a project's cmake invocation.


>> But that's maybe not the only glitch. I tried to import one of those MacPorts projects, configured as usual but with the "CodeBlocks - Unix Makefiles' generator under the assumption that Creator would be able to infer the project structure from that.
>
>Is one of these projects open-source? Can I open such projects on
>Linux for testing? I'd like to give this a whirl:-)

They're all open source. How much time do you want to invest in whirling? You may have understood by now that doing it really in the same context will require a "certain" investment in setting things up.
But it should be possible to approximate my full set-up. I used a simple KF5 Applications project for testing (KRuler), but I'd expect you should be able to use a simple Tier1 framework that doesn't have lots of additional dependencies, too. I can send you a trimmed-down script that invokes cmake directly with the really required arguments. If you invoke that to create a shadow build dir that sits next to the source dir you should get the same kind of information I have on my end and that don't import nicely.

>So you almost certainly have server-mode in your cmake:-)

Yes, but as evoked above, there are apparently certain issues with server mode that make it incompatible with what I'm trying to do, until v3.9 comes out.

>The CodeBlocks part is optional for you (your cmake has
>server-mode;-). So -G "Unix Makefiles" is ok. You can define the
>generator in the Kit.

Come to think of it, I have a hunch that a server mode might be incompatible with the kind of workflow described above. I'm tapping into a build system that calls on cmake but doesn't expect cmake to keep running as some sort of server instance.

>It has a section for CMake Configuration settings. That configuration

Yeah, I noticed. I think it could do with some helpful tooltips ;)

>> When I import the resulting build dir in Creator I don't see the source in the IDE, but install tree.
>
>What does that mean exactly? You do not see the files in the source
>directory but a list of files as they are installed?

Indeed. Rather than seeing the source files, I see the /opt/local tree. Very curious.

>I have considered using compile_commands.json file, but that would
>just be another hack that will fail for non-trivial projects. You get
>the exact compiler call for all source files from that file, but there
>is no context information in it, so you can not say which call belongs
>to which target. That is problematic when you build the same file
>several times -- something that happens a lot in the wild, e.g. in
>tests.

Hmm, that might expain why KDevelop doesn't always show the build targets, and indeed there's confusion sometimes when a file belongs to multiple targets. 
I think though you're not entirely right: AFAIU CMake creates a subdirectory per target, and puts the object file created from source files in the that CMakeFiles/TargetName.dir directory . Isn't that enough information to infer the targets, and determine which file belongs to what target(s)?

Cheers,
René