[Development] Notes on "Qt Build Systems" @ QtCon 2016

[Stephen Kelly]

Christian Kandeler wrote:

> [Sorry about the formatting, using outlook]
> 
> Stephen Kelly wrote:
>> Here's the CMake version:
> 
> [ ... ]
> 
>>      execute_process(
>>       COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/generator.py --list
>> ${CMAKE_CURRENT_BINARY_DIR}/genoutput 5
>>       OUTPUT_VARIABLE fileList
>>     )
> 
> How do you know where the input file is located?

There is no input file. There is only an input number. The task is from Bo, 
who gave it as a simplified example.
 
>> However, it is cheating in the same way that the QBS version from
>> Christian is cheating - it assumes '--list' exists:
> 
> Yes, I was assuming a cooperating tool.

In his real-world example, Bo said that there was a tool which generated 
output files, but the output files are not a-priori known: 

 https://www.mail-archive.com/development@qt-project.org/msg25970.html

Of course, if you have a 'cooperating tool' you can do anything with CMake 
too. My understanding is that because Qbs has a dynamic build graph, it can 
do things that CMake can't do. 

I'm looking for a good example of that so that I can understand. It is the 
gap that Qbs is to fill. Jake mentioned examples to show the difference 
between the Rolls-Royce Trent 900 jet engine that is Qbs, and the wet 
firecrackers that are CMake and qmake. I thought this would be a good one, 
but it turns out that in this case, Qbs and CMake have the same requirement 
of the tool.

>> Christian, can you create a version which does not require --list?
> 
> There are two possibilities:
> a) The inner workings of the tool are known and "simple enough". That's
> the case in Bo's example, so there we could just open the input file and
> derive the output artifacts from the number we find there. 

Yes, if we can find out "now" what the tool will generate "later", then 
everything is easy. 

If we require that of the tool, then we can have a static build graph, so I 
don't see how the dynamic build graph is an advantage in that case.

In the case Bo mentioned, he built the tool, so he can ensure it is 
cooperative. It seems his mistake and the source of his problem was to use 
qmake for the job instead of CMake or Qbs :).

> b) Otherwise,
> our outputArtifacts script has to run the tool in "real mode" (using the
> "--generate" option).

And how do you know what the outputArtifacts are? Do you require the tool to 
tell you what files it generated during this step? (ie, again be 
cooperative)

> The actual command would be a no-op then. This is
> icky, both conceptually and for practical reasons, because commands of
> non-competing rules are run in parallel, whereas the outputArtifacts
> scripts are not.

Right, assuming the tool is still 'cooperative' and tells you what files it 
generated, you would end up with the same thing with a CMake build - you 
would generate the files at CMake time, and then ninja would build them in 
parallel.

Again, that's no conceptual advantage of the dynamic build graph over the 
static one in the case of a somewhat cooperative tool. That's probably why 
Milian never saw a need for such a thing :).

So, I'm still trying to find out when the dynamic build graph is actually an 
advantage. I have an idea in mind, but it gets quite niche and still has 
some assumptions. Maybe you can tell me if I am right:

1) You have an 'uncooperative' tool which can not tell you what it will 
generate, and which does not output a list of what it generated. However, it 
accepts a command line argument specifying where it should generate its 
files.

2) So, you set it up to generate the files in some directory. You assume 
that the directory is empty before the tool is run, and you use the 
equivalent of running the 'ls' command to determine what the files are. 
Rules to run moc and to run the compiler are then added to your dynamic 
build graph, which then proceeds and eventually gets around to executing 
those rules.



If that is something Qbs is designed to do, then can you post a Qbs file 
which does it? You can use the generator.py I posted before. The script 
already accepts a directory to generate the files into - just pretend --list 
doesn't exist.

Thanks,

Steve.