[Development] Qt branches & proposal how to continue with those

[Olivier Goffart]

Am Freitag, 9. Februar 2018, 08:13:01 CET schrieb Thiago Macieira:
> On Thursday, 8 February 2018 23:02:36 PST Kevin Kofler wrote:
> > IMHO, you need to rethink your whole CI approach. This is increasingly
> > being the one bottleneck slowing down Qt development and releases. It
> > might make more sense to try a different approach, such as allowing all
> > commits through initially, then making CI runs at regular intervals, and
> > triggering reverts if things broke.
> 
> Which will happen ALL the time. We'll never get back down: when we released
> Qt 4.2, 4.3 and 4.4, we were happy if only 10 tests failed (that only
> happened for QWS). For the other platforms, the normal number was a hundred
> tests failing. Then we spent weeks trying to get the number down.
>
> Sorry, I don't want to go back to that.

That's not exactly how I remember it from the Qt 4.6 and 4.7 times where there 
was no CI yet, but we were actively fixing tests. There were always a few red 
tests on testr.troll.no, but they were mostly flacky tests. What is it 
compared to now with about 100 BLACKLIST files in qtbase alone? (there was no 
blacklist at the time)
 
That said, I agree we should not go back to that.

> 
> > Qt is being developed very much as a corporate project. (I write "as"
> > rather than "like" because that's what Qt is, despite Open Governance.)
> > It would help to look at how community Free Software projects do things.
> > They tend to be more efficient. And some company-developed Free Software
> > projects have already adopted such processes.
> 
> It would not do us well to look at poorer practices than what we have. Just
> because everyone else is where we were 10 years ago is no reason for us to
> go back to it. Show us a *better* model, one that still prevents failures
> from being added, and we'll consider it.

How pretentious are you, thinking that everybody else has poorer practice?
Other projects still manage to release product with passing test. And might 
not get ludicrous release delay "because of the CI system" that was supposed 
to help, but gets in the way instead.

Anyway, here is some example of models which works:

In LLVM, devs commit directly. buildbots build the trunk continuously. In case 
of failure, the buildbot maintainer quickly find out which commit likely broke 
the test and reverts the commit imediatly (or commit a fix if the fix is 
trivial)
This works because there are buildbot maintainers taking care of the build 
status, and they are not afraid to revert. Also the build and test time is 
reasonable (while still being big), and individual developer can easily run 
all the tests on their machine before submitting patches. (Very few platform 
specific tests).

On other projects I've seen on github with travis and co.:  the tests are run 
for every pull request individually, before it is integrated. The reviewer 
sees the result of the tests and decide whether to merge on not based on the 
result. If one sees that the failures is obviously unrelated to the patch, the 
reviewer will override and merge anyway.

I think this model could be easily applied to Qt.

In particular, there should be an option to merge a patch directly without 
going through CI. (If you are scared of abuse, limit it to admin or 
maintainers).
This could be used (exceptionally) for urgent patches such as patches that 
fixes the CI and that are not being integrated quickly enough because of other 
unrelated failures, continuing to block the CI for days. That way, the CI 
would not get in the way.

In Summary: The CI should be a tool helping the development, and not slowing 
it down. And having a way to override the CI for important patches should be 
an easy and quick way to improve things a bit.

-- 
Olivier

Woboq - Qt services and support - https://woboq.com - https://code.woboq.org