[Development] Focusing bug fixes to 5.9 branch and patch releases during H1/17
Sean Harmer
sean.harmer at kdab.com
Fri Feb 17 10:48:28 CET 2017
On Friday 17 February 2017 08:10:55 Alex Blasche wrote:
> > [...]
> >
> > I note that an answer on this is still pending, but as an aside: CI on
> > 5.8 (and 5.6) appears to be smooth sailing the last two days. Hardly any
> > false failures anymore.
>
> The CI integration process is only partly to blame. Yes, after each fix
> round there is a new round of qt5 git integrations with more unit test
> fixes followed by a new round of package generation.
> > I do realise that the main load probably comes from qt5.git
> > integrations, but even so, if 5.8 (qtbase) integrations run through the
> > first time instead of requiring half a dozen restages, either the CI
> > load is lowered significantly, helping to avoid flakiness in other
> > tests, or allowing more stuff to be merged per unit time.
>
> Each of the packages must be tested. We are talking about 17 different types
> of packages. For each package the tester has to install the package and run
> a series of tests. This means you have to have testers for each platform
> and this takes half a day to do. We are talking about principle testing
> for each feature domain, QtCreator, app install & deployment, completeness
> of the install package etc. it is not a sexy task either as it is very
> mundane.
Some steps of this can be automated with squish or similar but I appreciate
that may not be the case at the current time.
> This also assumes that each tester has time to do the testing at the same
> time. Whenever the last platform tester finishes that's when you can
> progress to the next step.
And this is one of the areas which KDAB is offering to help with if it gets a
5.8.1 release out. But we can only do this if the candidate packages are made
available.
> 5.8.1 is a release with a lot of changes. It would not be a low effort
> release testing. I do not believe that smoke testing is sufficient. At
> least the past .1 have very much proven this fact.
Sure, but 5.9.0 has even more changes and is therefore more of a risk of
taking longer. Nobody is debating that making a .1 release is not a lot of
effort. What we are arguing is that making 5.8.1 is more important than
releasing 5.9.0 a few weeks earlier than if there is a 5.8.1.
As makers of a toolkit there is an implicit promise that there will be timely
bug fix releases. It seems that The Qt Company has decided otherwise and that
releasing 5.9.0 before the summer vacations is more important.
At the end of the day, all we, KDAB and other community members, can do is
point out that your customers may well not see it this way and offer to help.
But since you control all of the infrastructure there is little else we can do
about it.
> Yes, there is things we can and want to improve but we are not there yet. I
> also do not believe that making a bad release is helping customers either.
Again, nobody wants a bad release. We're suggesting a different prioritisation
of releases, not sacrificing on quality for any of them.
Cheers,
Sean
> > Shutting down 5.8 because of load problems in the CI now makes even less
> > sense than before.
>
> As shown above it is not.
> --
> Alex
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
--
Dr Sean Harmer | sean.harmer at kdab.com | Managing Director UK
KDAB (UK) Ltd, a KDAB Group company
Tel. +44 (0)1625 809908; Sweden (HQ) +46-563-540090
Mobile: +44 (0)7545 140604
KDAB - Qt Experts
More information about the Development
mailing list