[Development] Integration of qt5.git
Jani.Heikkinen at digia.com
Mon Dec 2 08:20:42 CET 2013
I agree we need to do something for this. But is it really OK to remove all CI testing from qt5.git integration because even there has been lots of false failures there has been correct ones as well? In my opinion we should first take a look into current CI setups, remove all insignificants, force success etc and when those are OK then we can proceed with this... Even this is delaying our work it isn't correct approach to do less (blocking) testing.
From: development-bounces+jani.heikkinen=digia.com at qt-project.org [mailto:development-bounces+jani.heikkinen=digia.com at qt-project.org] On Behalf Of Knoll Lars
Sent: 30. marraskuuta 2013 17:04
To: Gladhorn Frederik; Hausmann Simon; development at qt-project.org
Subject: Re: [Development] Integration of qt5.git
On 30/11/13 15:22, "Gladhorn Frederik" <Frederik.Gladhorn at digia.com<mailto:Frederik.Gladhorn at digia.com>> wrote:
On Friday 29. November 2013 21.19.33 Hausmann Simon wrote:
> Once scenario I'm worried about is when a less active module (say qtscript)
> has a regression test failure due to a change in an active module (qtbase)
> and it's only visible in the qt5.git integration (because nobody submits a
> change to qtscript).
> We will see the failure still, but we will also be very very tempted to
> still cut a release from qt5.git because of schedules etc. , despite the
> Currently we have a mechanism in place that makes it very hard to release in
> such a situation. We are removing this protection and am replacing an
> automated mechanism with a human gate keeper.
I completely agree with your concern. Actually I was arguing in a similar direction before. I'd still like to try this.
My argument is that before it could be a few days until anyone ever tried updating qt5.git and there was no system doing this automatically on a regular basis.
When we broke something in qtscript by changing qtbase, so far we only noticed several days later. I'm not sure if the nightly build makes the situation much better, but imho it doesn't get worse either.
I'm also in very much favor of trying this. We're seeing that the current approach simply doesn't work. I'd be almost saying that the value of running full CI rounds on qt5.git has IMO non existant to negative. It takes too long to get updates through and many integrations fail due to unstable tests. qt5.git integrations are making packaging and releasing harder and that's the main thing they are being used for. So it's time to try solving this differently.
If we are worried about cross dependencies between projects I think we're better off adding a one or two more revdeps where we really need them. The advantage of revdeps is also that they run in parallel to the other platform tests, and thus don't slow down CI as much as the full (serial) run of all tests on qt5.git.
> I'm not against trying this, but we have to be very careful I think, more
> discipline will be required from the release side or else we will release
> with known regressions.
> Fra: Gladhorn Frederik
> Sendt: 18:16 fredag 29. november 2013
> Til: development at qt-project.org<mailto:development at qt-project.org>
> Emne: [Development] Integration of qt5.git
> Hello all,
> I would like to get feedback about a change in the qt5.git integration that
> I have discussed with several people in Digia's Oslo office.
> For those that don't know what I'm talking about: we have the meta
> repository qt5.git that has all other modules as git submodule. When you
> clone qt5 and run the init-repository script it checks out the submodules
> according to the current state of qt5.git.
> Currently we have a bot doing merges and someone needs to approve them.
> Once a merge (this is per branch, release/stable/dev) is approved it gets
> run though CI, just like any other patch. Then all modules are built and
> all tests run. When all tests pass we have the submodules updated and the
> game starts anew.
> The good thing is that usually qt5.git is in a state where it works and it's
> tested. One issue is that most people working on Qt have all branches
> checked out to release/stable/dev to a newer state and work on that. But
> more importantly we need qt5.git to be updated for the release. Usually
> updating qt5.git works, but sometimes it turns out to take a lot of time
> and is major work. As you may have noticed it takes a long time to update
> the packages and this is one factor contributing to that.
> I'd like to propose the following:
> Automate qt5.git integration completely (the bot updates the modules and
> instantly stages the update), this could happen around 3 times per day.
> This integration would not run any tests and should generally just succeed
> unless we break compilation of one modules by changing it's dependencies
> (happens very seldom). Failures send email to this mailing list.
> Since we lose the test runs for all modules at the same time I'd like to
> have a second and new job in Jenkins that runs nightly and builds and runs
> all tests of all modules (according to qt5.git state). This job doesn't
> mean anything for the integration and doesn't block but simply sends mail
> to this list whenever any test fails. I actually expect it to fail
> regularily since we have quite a few unstable unit tests that every once in
> a while fail. I hope that this approach gives actually more visibility to
> failing auto tests than what we currently have.
> I hope all in all this lets us move faster and get releases out with less
> pain and stress since it's faster to include a patch or two in the release
> branch. And it lessens the different experiences people have when checking
> out qt5.git.
_______________________________________________ Development mailing list Development at qt-project.org<mailto:Development at qt-project.org> http://lists.qt-project.org/mailman/listinfo/development
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Development