[Development] Proposal to adjust release candidate process

[Jani Heikkinen]

I don't see any more testing required with this new approach than earlier because earlier we created snapshots to be tested, now we will do additional betas.
The thing which will change is the way we will do the testing: Earlier we need to wait until all packages were tested manually before release but with this new approach we rely on RTA. Manual testing will mostly start after the beta n release; if something is broken in the beta n it can be fixed in beta n+1 but still well before final release. 

br,
Jani 

> -----Original Message-----
> From: Maurice Kalinowski
> Sent: tiistaina 28. maaliskuuta 2017 10.33
> To: Jani Heikkinen <jani.heikkinen at qt.io>; Lars Knoll <lars.knoll at qt.io>; Alex
> Blasche <alexander.blasche at qt.io>
> Cc: Qt development mailing list <development at qt-project.org>
> Subject: RE: [Development] Proposal to adjust release candidate process
> 
> Releasing beta more frequently is a welcomed item, for sure.
> 
> However, that implies that there is even more packages to test and what has
> not been discussed is the fact that there are not more people testing.
> 
> For WinRT the situation has been that Olli and I needed to test
> winrt/winphone-(arm/x64)-msvc(2013/2015), and that at least once per
> week. As you can imagine, if you go through the testing matrix properly that's
> around half a day for one package with multiple packages per person. There
> is not much time left to do development and bug fixing. For other platform
> teams the situation is the same. This has the effect that you start to be more
> "flexible" in what you test and hence it is way easier for a bug to slip through.
> Certainly we want to avoid that, but have not tackled any way to do so.
> 
> I remember that others volunteered to help with package testing. That surely
> would be welcomed, but also given the fact that Jani sends out testing
> requests fairly often and the response rate is zero / nada / null / nil since
> years, I doubt that will change as well.
> 
> Preferrably I'd not like to spend again 50%+ of my time testing packages until
> summer, or even more once the 5.8.1 discussion comes up again.
> 
> 
> Maurice
> 
> 
> > -----Original Message-----
> > From: Development [mailto:development-
> > bounces+maurice.kalinowski=qt.io at qt-project.org] On Behalf Of Jani
> > Heikkinen
> > Sent: Tuesday, March 28, 2017 8:48 AM
> > To: Lars Knoll <lars.knoll at qt.io>; Alex Blasche
> > <alexander.blasche at qt.io>
> > Cc: Qt development mailing list <development at qt-project.org>
> > Subject: Re: [Development] Proposal to adjust release candidate
> > process
> >
> >
> > > -----Original Message-----
> > > From: Development [mailto:development-
> > bounces+jani.heikkinen=qt.io at qt-
> > > project.org] On Behalf Of Lars Knoll
> > > Sent: tiistaina 28. maaliskuuta 2017 9.33
> > > To: Alex Blasche <alexander.blasche at qt.io>
> > > Cc: Qt development mailing list <development at qt-project.org>
> > > Subject: Re: [Development] Proposal to adjust release candidate
> > > process
> > >
> > >
> > > > On 28 Mar 2017, at 08:09, Alex Blasche <alexander.blasche at qt.io>
> wrote:
> > > >
> > > >
> > > >>> Sounds good, but please remember we'll need to keep at least the
> > > >>> source tarballs for each and every beta and RC that we release.
> > > >>
> > > >> Yeah, that's OK
> > > >
> > > > This implies that the user can identify the exact tag or version
> > > > (aka beta 5)
> > > somewhere in the installer. Each bug report has to state this value.
> > >
> > > Yes, either through a number or (preferable IMO) by simply putting
> > > the
> > > sha1 of qt5.git into a visible place.
> >
> > We were planning to use Beta N in installer UI as well as git tags.
> > should be ok and it is following current ways of working
> >
> > br,
> > Jani
> >
> >
> > >
> > > Cheers,
> > > Lars
> > >
> > > _______________________________________________
> > > Development mailing list
> > > Development at qt-project.org
> > > http://lists.qt-project.org/mailman/listinfo/development
> > _______________________________________________
> > Development mailing list
> > Development at qt-project.org
> > http://lists.qt-project.org/mailman/listinfo/development