[Development] Focusing bug fixes to 5.9 branch and patch releases during H1/17

Sean Harmer sean.harmer at kdab.com
Fri Feb 17 10:48:28 CET 2017


On Friday 17 February 2017 08:10:55 Alex Blasche wrote:
> > [...]
> > 
> > I note that an answer on this is still pending, but as an aside: CI on
> > 5.8 (and 5.6) appears to be smooth sailing the last two days. Hardly any
> > false failures anymore.
> 
> The CI integration process is only partly to blame. Yes, after each fix
> round there is a new round of qt5 git integrations with more unit test
> fixes followed by a new round of package generation.
> > I do realise that the main load probably comes from qt5.git
> > integrations, but even so, if 5.8 (qtbase) integrations run through the
> > first time instead of requiring half a dozen restages, either the CI
> > load is lowered significantly, helping to avoid flakiness in other
> > tests, or allowing more stuff to be merged per unit time.
> 
> Each of the packages must be tested. We are talking about 17 different types
> of packages. For each package the tester has to install the package and run
> a series of tests. This means you have to have testers for each platform
> and this takes half a day to do.  We are talking about principle testing
> for each feature domain, QtCreator, app install & deployment, completeness
> of the install package etc. it is not a sexy task either as it is very
> mundane.

Some steps of this can be automated with squish or similar but I appreciate 
that may not be the case at the current time.

> This also assumes that each tester has time to do the testing at the same
> time. Whenever the last platform tester finishes that's when you can
> progress to the next step.

And this is one of the areas which KDAB is offering to help with if it gets a 
5.8.1 release out. But we can only do this if the candidate packages are made 
available.

> 5.8.1 is a release with a lot of changes. It would not be a low effort
> release testing. I do not believe that smoke testing is sufficient. At
> least the past .1 have very much proven this fact.

Sure, but 5.9.0 has even more changes and is therefore more of a risk of 
taking longer. Nobody is debating that making a .1 release is not a lot of 
effort. What we are arguing is that making 5.8.1 is more important than 
releasing 5.9.0 a few weeks earlier than if there is a 5.8.1.

As makers of a toolkit there is an implicit promise that there will be timely 
bug fix releases. It seems that The Qt Company has decided otherwise and that 
releasing 5.9.0 before the summer vacations is more important.

At the end of the day, all we, KDAB and other community members, can do is 
point out that your customers may well not see it this way and offer to help. 
But since you control all of the infrastructure there is little else we can do 
about it.

> Yes, there is things we can and want to improve but we are not there yet. I
> also do not believe that making a bad release is helping customers either.

Again, nobody wants a bad release. We're suggesting a different prioritisation 
of releases, not sacrificing on quality for any of them.

Cheers,

Sean

> > Shutting down 5.8 because of load problems in the CI now makes even less
> > sense than before.
> 
> As shown above it is not.
> --
> Alex
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

-- 
Dr Sean Harmer | sean.harmer at kdab.com | Managing Director UK
KDAB (UK) Ltd, a KDAB Group company
Tel. +44 (0)1625 809908; Sweden (HQ) +46-563-540090
Mobile: +44 (0)7545 140604
KDAB - Qt Experts



More information about the Development mailing list