[Development] Qt modules, API changes and Qt 6

Simon Hausmann Simon.Hausmann at qt.io
Tue Sep 17 08:56:39 CEST 2019


I would like to provide an update to this (old) thread based on the 
development in the past months and weeks.

Am 15.02.19 um 10:30 schrieb Frederik Gladhorn:
> Hi,
> On fredag 15. februar 2019 07:31:33 CET Lars Knoll wrote:
>> Summing up the discussion here. It looks like people overall agree that the
>> pinned dependency approach (option 3) sounds better than what we currently
>> have. The main concern was CI capacity, but Frederik believes that with
>> enough storage capacity for build artefacts this will not be worse than
>> what we have today with all the failed qt5.git integrations.
>> The only other option that was seriously considered was a more monolithic
>> repo (option 4). Disadvantages here are that it would require more work on
>> the CI front to make testing times bearable, and it doesn’t give us a very
>> flexible approach with regards to adding/removing modules. This is
>> something that option 3 will nicely give us, as it treats all repositories
>> the same way. Option 3 does btw not exclude that we later on merge some
>> repositories if we feel that makes sense.
>> So let’s go with option 3. Frederik will be leading the work to get this
>> implemented.
>> As far as I can see, it requires some changes to CI so that we have the
>> dependencies encoded in each repository, a bot to automatically push sha1’s
>> of dependencies forward and some monitoring to alert us if modules get left
>> behind.
> I have indeed been playing with this, and I think I have an implementation
> that works when it comes to resolving dependencies based on the idea.
> After some pondering and starting with ini files, I settled on yaml, since we
> probably want to extend the data later on and I would like this to be easily
> human and machine read- and writable.
> My current files look like this:
>   dependencies:
>       qt/qtbase:
>           ref: 698078680fc5a6870177af285fa50c0d8a7c0bc3
>           url: code.qt.io
>       qt/qtsvg:
>           ref: 2ae3c52fc061f574c9582bf58963fb3996724fbf
>           url: code.qt.io
> This is for qt/qtdeclarative, sha1's taken from qt5.git at some point.
> Then there is the question of what the file name should be. Should it be at
> the top-level or in a coin subdirectory? dependencies.yaml?
> I have not yet worked on the bot doing the module updates (which may turn out
> to be the harder part).
> I don't think this can go live before we have that written as well.

Thanks to Frederik and Aapo, Coin has learned to read and utilize the 
dependencies.yaml files in repositories when present. This permits 
pinning dependencies to specific sha1s.

The second part was a tool to maintain these permanent pinnings, by 
posting updates to Gerrit through the graph of dependencies. This tool 
is now ready for use and is based on a fairly simple algorithm. I'd like 
to put this tool into Gerrit in the near future. Meanwhile, here's a 
description of how it works from the README:

## Algorithm

The process of updating dependencies starts by collecting a list of all 
repositories and determining the root of the graph. That's typically 
qtbase. From there one, updates to all repositories are posted that only 
depend on the root. All other repositories remain in a "todo" list. The 
root is remembered in a "done" list and all repositories that we are 
currently trying to bring up-to-date are in a "pending" list. Once this 
process is started, the program saves its state in a ```.json``` file 
and terminates.

The next time the Qt Module Updater is started, it resumes the state and 
begins checking the state of all pending updates. If an update 
succeeded, then the corresponding repository is added to the "done" list 
and we can prepare updates for repositories that have now their 
dependencies satisfied by picking them from the "todo" list. If the 
update failed, the repository is dropped from the batch of updates and 
all other repositories that directly or indirectly depend on the failed 
one are also removed. After every such iteration of processing pending 
updates and pushing new ones to Gerrit, the process terminates and saves 
its state.

When the todo list is empty and there are no more pending updates, the 
batch update is complete. If during that update there were no failures, 
the Qt Module Updater will also push a change to qt5.git with an update 
to all submodule sha1s of the new consistent set of modules.

Since this is needed to facilitate private API changes in Qt 6, the 
objective is to take this into production in either the wip/qt6 branch 
or dev in the coming days. If you see these files appearing, then you 
know why :-)

There is no immediate plan to take this into use for Qt 5 development. 
That's not a "never ever!" statement though, more an expression of 
"let's try this in Qt 6 first".


More information about the Development mailing list