[Development] Qt modules, API changes and Qt 6

[Edward Welbourne]

Robin Burchell (30 January 2019 10:13)
> I will admit that a monorepo has a _different_ set of problems
> (including but not limited to: longer build times, longer test times,
> flaky tests in unrelated areas blocking changes),

It also makes it easier to cope with API changes, which is great where
it's public APIs that haven't yet been shipped, but also makes it easier
to get away with using private APIs between components that really
shouldn't do that.  One of the classic reasons for modularisation at the
VC layer is that it makes this sort of thing harder, which means it
happens less, which is good.

There's also the problem of scope - which things go in the monorepo,
which should be outside.  We have that today, with qt5/, and we should
probably hoist some of its pieces outside, if only to force ourselves to
make it easy for a sizeable component to live happily outside; that
would enable folk in our ecosystem to live happily alongside, rather
than inside, Qt.  If we insist on solving that as part of a switch to a
monorepo, then we win (even if we could have done it without the
switch), if only because a major upheaval is an opportunity to make
other needed changes.  But if we move to a monorepo without solving that
problem, there's a significant risk we'll be making things harder for
those who work outside but close to Qt.

> but those problems are not complex, and can be fixed with some
> dedicated application of smarter scripting at build/test time

I remain to be convinced.

> (for instance: if change is doc only, don't run any test that _isn't_
> related to documentation, to cover one complaint from earlier in this
> thread).

This sort of thing [*] sounds terribly sensible and feasible, until you
start running into changes that the submitter and reviewers all *think*
should only have impact in a bounded area, but that turns out to break
stuff in surprising places outside those bounds.  That's probably rare
but when it happens it'll gum up the works - in a seemingly not very
related area that's been caught in the cross-fire.  In particular, this
sort of thing happens more readily when disparate things use each
others private APIs, as sketched above.

[*] The case of doc fixes is probably relatively safe, of course; but if
this is applied to other changes, we can't be assured of as much safety.
One of the scripts involved in my API change review generator knows to
ignore various changes that "make no difference"; we could apply
something like that to changes to say "needs minimal testing"; but I'd
still worry about the cases where a change makes more difference than
the script maintainer is aware of.  Once we get to "this only changes
network code, we don't need to test graphics" (or vice versa) you can
start to expect sporadic surprises.

Not that the present state of affairs entirely avoids that situation.
In commit qtbase's 641eb4a965 I reverted the introduction of GPU
blacklisting, since it's no longer used; in the process, I renamed
QTestPrivate::checkBlackLists() back to the singular name it'd had
before GPU blcaklists were introduced, confidently expecting that to
have no impact outside QtTestLib.  That broke qtdeclarative, because it
actually uses this private API (in implementing the QML test framework),
resulting in a crisis that Liang fixed with qtbase's af6d4d068.  That
would have been avoided by a monorepo, but not if we were only building
and testing the parts we believed should be affected.

So we need to be deliberate about refraining from and objecting to
cross-component use of private parts, all the more so if we're going to
a monorepo.  We should also document, alongside each private API, any
known violations of its privacy; normally, those are done by friend
declarations, but non-class cases (like the QTestPrivate namespace) need
comments about such things (and those comments need to be specific
enough that someone finding them years later can determine whether
they've gone out of date).

	Eddy.