[QBS] Rule generating lots of files

[Jake Petroules]

> On Aug 26, 2015, at 9:07 AM, Christian Kandeler <christian.kandeler at theqtcompany.com> wrote:
> 
> On 08/26/2015 05:24 PM, Stephan Gatzka wrote:
>> My problem is, that I can't specify wildcards in the outputArtifacts
>> property, so currently I only see to specify every single file by hand
>> in outputArtifacts, which is rather tedious and in addition might change
>> because depending from the inputs.
>> 
>> Is there a clever way to achieve this? A workaround might be to generate
>> a list of C files in the prepare script and hand over that list to
>> outputArtifacts?
> 
> The outputArtifacts script is called first, and the prepare script gets 
> its information about what the outputs are from there.
> For the case where the Rule cannot know in advance which outputs will be 
> generated (e.g. because that depends on the contents of input files 
> and/or the logic to derive the file paths would duplicate lots of code 
> from the tool to run), there are the following approaches:
>     a) Ideally, the tool provides some sort of "dry run" mode where it
>        only generates the information about which files it would create.
>        You would call the tool in that mode in the outputArtifacts
>        script.
>     b) You call the tool twice. The first run happens in the
>        outputArtifacts script and is only used to find out which
>        files were generated. Can be tolerable if running the tool is
>        cheap.
>     c) If b) is too expensive, you can cheat and do the actual work
>        in the outputArtifacts script, making the prepare script
>        a no-op. I think Jake Petroules has worked on a somwhat cleaner
>        version of this approach recently: he might want to chime in
>        here.

There is a JavaScript class in ModUtils called BlackboxOutputArtifactTracker. Using this involves a few steps:

Run the tool in the outputArtifacts script which will allow the tracker to collect the list of file paths that were created
Supply the destination directory you *would* compile to, to the tracker
Supply a callback to the tracker to add the appropriate file tags to these artifacts based on their file paths
Run the tool as you normally would in the prepare script using the Command class

This is the (b) scenario that Christian mentioned above. While this sounds somewhat convoluted, it's necessary in order to properly track all files and their dependencies in the qbs build graph. I also have a pending optimization to allow the generated artifacts to be moved into place (renamed) to the destination paths semi-automatically in order to reduce build time by running the tool only once instead of twice (similar to scenario (c) but doesn't have the disadvantage of modifying files corresponding to build graph nodes in what *should* be a read-only operation).

There are examples of the tracker's usage in IBModule.qbs + ib.js and core.qbs + qdoc.js -- the latter's is probably the most straightforward reference -- but please note this API is considered internal and is not guaranteed to maintain compatibility in the future (or even exist at all), pending a significantly better and more flexible approach built directly into the qbs architecture. That said, it's unlikely to change any time soon.

If you have the ability to modify this tool and add a dry run mode as mentioned in (a), that would be the best course of action to take.

> 
> 
> Christian
> _______________________________________________
> QBS mailing list
> QBS at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/qbs

-- 
Jake Petroules - jake.petroules at petroules.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/qbs/attachments/20150826/28a3d14c/attachment.html>