[Development] Qt modules, API changes and Qt 6

[Lars Knoll]

Summing up the discussion here. It looks like people overall agree that the pinned dependency approach (option 3) sounds better than what we currently have. The main concern was CI capacity, but Frederik believes that with enough storage capacity for build artefacts this will not be worse than what we have today with all the failed qt5.git integrations.

The only other option that was seriously considered was a more monolithic repo (option 4). Disadvantages here are that it would require more work on the CI front to make testing times bearable, and it doesn’t give us a very flexible approach with regards to adding/removing modules. This is something that option 3 will nicely give us, as it treats all repositories the same way. Option 3 does btw not exclude that we later on merge some repositories if we feel that makes sense.

So let’s go with option 3. Frederik will be leading the work to get this implemented. 

As far as I can see, it requires some changes to CI so that we have the dependencies encoded in each repository, a bot to automatically push sha1’s of dependencies forward and some monitoring to alert us if modules get left behind.

Cheers,
Lars

> On 30 Jan 2019, at 11:11, Edward Welbourne <edward.welbourne at qt.io> wrote:
> 
> Robin Burchell (30 January 2019 10:13)
>> I will admit that a monorepo has a _different_ set of problems
>> (including but not limited to: longer build times, longer test times,
>> flaky tests in unrelated areas blocking changes),
> 
> It also makes it easier to cope with API changes, which is great where
> it's public APIs that haven't yet been shipped, but also makes it easier
> to get away with using private APIs between components that really
> shouldn't do that.  One of the classic reasons for modularisation at the
> VC layer is that it makes this sort of thing harder, which means it
> happens less, which is good.
> 
> There's also the problem of scope - which things go in the monorepo,
> which should be outside.  We have that today, with qt5/, and we should
> probably hoist some of its pieces outside, if only to force ourselves to
> make it easy for a sizeable component to live happily outside; that
> would enable folk in our ecosystem to live happily alongside, rather
> than inside, Qt.  If we insist on solving that as part of a switch to a
> monorepo, then we win (even if we could have done it without the
> switch), if only because a major upheaval is an opportunity to make
> other needed changes.  But if we move to a monorepo without solving that
> problem, there's a significant risk we'll be making things harder for
> those who work outside but close to Qt.
> 
>> but those problems are not complex, and can be fixed with some
>> dedicated application of smarter scripting at build/test time
> 
> I remain to be convinced.
> 
>> (for instance: if change is doc only, don't run any test that _isn't_
>> related to documentation, to cover one complaint from earlier in this
>> thread).
> 
> This sort of thing [*] sounds terribly sensible and feasible, until you
> start running into changes that the submitter and reviewers all *think*
> should only have impact in a bounded area, but that turns out to break
> stuff in surprising places outside those bounds.  That's probably rare
> but when it happens it'll gum up the works - in a seemingly not very
> related area that's been caught in the cross-fire.  In particular, this
> sort of thing happens more readily when disparate things use each
> others private APIs, as sketched above.
> 
> [*] The case of doc fixes is probably relatively safe, of course; but if
> this is applied to other changes, we can't be assured of as much safety.
> One of the scripts involved in my API change review generator knows to
> ignore various changes that "make no difference"; we could apply
> something like that to changes to say "needs minimal testing"; but I'd
> still worry about the cases where a change makes more difference than
> the script maintainer is aware of.  Once we get to "this only changes
> network code, we don't need to test graphics" (or vice versa) you can
> start to expect sporadic surprises.
> 
> Not that the present state of affairs entirely avoids that situation.
> In commit qtbase's 641eb4a965 I reverted the introduction of GPU
> blacklisting, since it's no longer used; in the process, I renamed
> QTestPrivate::checkBlackLists() back to the singular name it'd had
> before GPU blcaklists were introduced, confidently expecting that to
> have no impact outside QtTestLib.  That broke qtdeclarative, because it
> actually uses this private API (in implementing the QML test framework),
> resulting in a crisis that Liang fixed with qtbase's af6d4d068.  That
> would have been avoided by a monorepo, but not if we were only building
> and testing the parts we believed should be affected.
> 
> So we need to be deliberate about refraining from and objecting to
> cross-component use of private parts, all the more so if we're going to
> a monorepo.  We should also document, alongside each private API, any
> known violations of its privacy; normally, those are done by friend
> declarations, but non-class cases (like the QTestPrivate namespace) need
> comments about such things (and those comments need to be specific
> enough that someone finding them years later can determine whether
> they've gone out of date).
> 
> 	Eddy.
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> https://lists.qt-project.org/listinfo/development