[Development] Focusing bug fixes to 5.9 branch and patch releases during H1/17

[Alex Blasche]

> -----Original Message-----
> From: Development [mailto:development-
> bounces+alexander.blasche=qt.io at qt-project.org] On Behalf Of Marc Mutz

> > Also, to allow others to help with the release process, could you
> > explain where the main bottle neck is with the release process please?
> > Is it the package generation itself? The smoke testing? Something
> > else?
> [...]
> 
> I note that an answer on this is still pending, but as an aside: CI on
> 5.8 (and 5.6) appears to be smooth sailing the last two days. Hardly any
> false failures anymore.

The CI integration process is only partly to blame. Yes, after each fix round there is a new round of qt5 git integrations with more unit test fixes followed by a new round of package generation.

 
> I do realise that the main load probably comes from qt5.git
> integrations, but even so, if 5.8 (qtbase) integrations run through the
> first time instead of requiring half a dozen restages, either the CI
> load is lowered significantly, helping to avoid flakiness in other
> tests, or allowing more stuff to be merged per unit time.

Each of the packages must be tested. We are talking about 17 different types of packages. For each package the tester has to install the package and run a series of tests. This means you have to have testers for each platform and this takes half a day to do.  We are talking about principle testing for each feature domain, QtCreator, app install & deployment, completeness of the install package etc. it is not a sexy task either as it is very mundane.

This also assumes that each tester has time to do the testing at the same time. Whenever the last platform tester finishes that's when you can progress to the next step.

5.8.1 is a release with a lot of changes. It would not be a low effort release testing. I do not believe that smoke testing is sufficient. At least the past .1 have very much proven this fact.

Yes, there is things we can and want to improve but we are not there yet. I also do not believe that making a bad release is helping customers either.

> Shutting down 5.8 because of load problems in the CI now makes even less
> sense than before.

As shown above it is not.
--
Alex