[Development] Qt 5.12 schedule proposal & proposal for release process change

Tue Apr 17 08:05:25 CEST 2018

> -----Original Message-----
> From: Development <development-bounces+jani.heikkinen=qt.io at qt-
> project.org> On Behalf Of Frederik Gladhorn
> Sent: maanantai 16. huhtikuuta 2018 15.11
> To: development at qt-project.org
> Subject: Re: [Development] Qt 5.12 schedule proposal & proposal for release
> process change
> 
> On mandag 16. april 2018 11.35.28 CEST Jani Heikkinen wrote:
> > Hi!
> >
> > Keeping release tags is OK for me. Motivation behind my proposal was
> > to stop planning other release dates than FF & final target. Currently
> > we are trying to estimate also alpha, beta and RC release dates and it
> > seems to cause some delays for us: persons aren't fixing blocker
> > issues or doing the test because it seems there is a time to do that
> > later (because e.g RC date is planned to be after a month). So if the
> > way to go is to do these "releases" immediately when ready it might
> > help us to get fixes in earlier and so on also get releases out quicker...
> 
> This sounds like it is the real issue. In my opinion, we should have a list of
> "known issues for betas" (does anyone know of a good way of tracking
> these, so that it's obvious and little work?). JIRA? Somewhere else?

Hmm, we have this already in jira; we just need to create a proper filter for it. And in addition we have that blocker list in the use where all release (==RC) blockers should be visible and which is used to track when we are ready for RC.

> 
> I think each beta is for finding more issues (if we consider it necessary), but
> we should not have anything block a beta from going out (OK, maybe if it
> doesn't compile, but how did we end up in such a bad state?). So there is the
> beta with a potentially growing and then shrinking list of release blockers.

Actually we are already working pretty much like this: We don't have a blocker list for beta n release. For beta1 we have and it is to make sure beta1 will be good enough. But after that we just put new beta n out when packages ready. And target is to get these out within 2 weeks. 

> 
> This means we may have critical bugs not fixed in the next beta, only in the
> RC latest.

As written above we don't have blocker lists for beta n; just for RC

> At the same time, I do wonder how strong the argument for more than one
> beta release really is. We should rather offer continuous post-beta
> snapshots, but IMHO we should only ask to test exactly once, during one
> beta.
> If we then trust that we fix the issues that were uncovered, we should move
> to the RC phase, without another beta (beta2/3/4/... contain very few
> changes compared to the first one anyway). There is little value in asking a
> great amount of people to re-test. The bad issues uncovered in the beta can
> be tested using post-beta snapshots, as needed.

Yeah, I agree this at least some level. I think it doesn't matter how the packages are called. And that's why I proposed to stop releasing these. But now I understand that most probably we need those milestones and phases. And so on it is most clear to call releases as alpha or beta... But I think it would be pretty confusing if we release beta1 and then beta snapshots. We have used snapshot term for releases which aren't ready yet (alpha snapshot == pre-alpha, beta snapshot == pre beta1 etc) and so on publishing beta snapshot after beta1 would be a somehow confusing. Ok it could be e.g 5.12 snapshot (after beta release) but I don't see that much sense to stop delivering beta n releases if we continue these alpha and beta phases: next delivery after beta(1) release is naturally beta2 etc.

And what comes to beta n changes: I wouldn't say there is very few changes in beta n compared to beta1.
5.11 alpha -> beta1:400 changes
5.11 beta1 ->beta2: 310 changes
5.11 beta2 ->beta3: 516 changes

So I would say beta n:s are as necessary as beta1...

> 
> The only thing that can be blocked imho is the release candidate. We should
> never have a release candidate that does not include fixes for all known
> issues. The release candidate should then turn into the final release after a
> brief sanity check. There must not be any changes from release candidate to
> release. If it turns out that we really messed up badly, we'll have another RC
> + release.

I totally agree with this. And this is what I have tried to do. With 5.10 this worked already quite well but still there is room for improvements: If we found some new blocker from RC(n) we should fix just it and nothing else. And then release RC n+1. 

And as I propose we should release RC immediately when blockers are fixed. And that's why I don't like to plan the date for it because it seems to slow down the process with fixing the release blockers.

> 
> I think this would help us get releases out quickly. It would mean, we really
> need to play by our own rules:
> after alpha: no more API changes, unless fixing new API. All new features
> must be _complete_, with no known big bugs (or thrown out again). I am
> certainly guilty of breaking this rule in the past and delaying releases...
> after beta: no more changes, except for fixes for release blockers release
> candidate: no more changes, unless brown paper bag issues are found

Yes please :D

Br,
Jani

> 
> Cheers,
> Frederik
> 
> 
> 
> >
> > But this change doesn't meant we need to stop calling those releases
> > as Alpha or Betas so we can continue labeling those as earlier.
> >
> > br,
> > Jani
> >
> > > -----Original Message-----
> > > From: Development <development-bounces+jani.heikkinen=qt.io at qt-
> > > project.org> On Behalf Of Lars Knoll
> > > Sent: maanantai 16. huhtikuuta 2018 10.03
> > > To: Simon Hausmann <Simon.Hausmann at qt.io>
> > > Cc: Qt development mailing list <development at qt-project.org>
> > > Subject: Re: [Development] Qt 5.12 schedule proposal & proposal for
> > > release process change
> > >
> > > Hi,
> > >
> > > From a releasing perspective I agree that the set we should update
> > > our packages often. We should also stop delivering source only alpha
> > > packages, and have binaries from the get go.
> > >
> > > But from a user perspective, I agree that it's good and important to
> > > put a name to those releases so people know what to expect from them.
> > > Snapshots as long as the packages are created from the dev branch,
> > > then alpha, beta, RC and final. We could discuss whether we need the
> > > alpha label, or whether we can go directly to beta, but I believe
> > > that we do need the other tags.
> > >
> > > But I do agree with releasing packages as often as possible, and we
> > > should have them already before FF from the dev branch (if possible).
> > >
> > > So I’d propose to have our packages updated often. But we do keep
> > > the ‘snapshot’, ‘alpha’, ‘beta’ and ‘RC’ tags.
> > >
> > > - Snapshot packages from the dev branch
> > > - FF and branching occur together and packages are getting the
> > > ‘alpha’ tag
> > > - Agree that we should start with the API review immediately.
> > > - Beta once API review is done
> > > - RC once we have the release branch and all currently known
> > > blockers are fixed
> > >
> > > That should be pretty close to what Jani described, only that we
> > > keep the tags for our users.
> > >
> > > Cheers,
> > > Lars
> > >
> > > > On 16 Apr 2018, at 08:43, Simon Hausmann <Simon.Hausmann at qt.io>
> > >
> > > wrote:
> > > > Hi,
> > > >
> > > > Same here. At first this seemed like a good idea, but it becomes
> > > > awkward
> > >
> > > when you look at it from the dev workflow point of view: After
> > > fixing a bug, what is the fix version? Reported in snapshot-24062918
> > > and fixed in snapshot-14072918? That sounds like it would create a
> > > mess in JIRA.>
> > > > Unless of course the task of assigning the fix version field was
> > > > automated.
> > > >
> > > > Simon
> > > >
> > > > On 16. Apr 2018, at 08:12, Alex Blasche <alexander.blasche at qt.io>
> wrote:
> > > >>> -----Original Message-----
> > > >>>
> > > >>>> On Thursday, 12 April 2018 01:42:29 PDT Jani Heikkinen wrote:
> > > >>>> And at this same time I want to propose that we stop delivering
> > > >>>> alpha or beta releases and just do snapshots instead.
> > > >>>> Publishing regular snapshots should be done until we are ready
> > > >>>> for RC. That because I don't see that much need for those
> > > >>>> anymore. Those are nowadays kind of milestones and in my
> > > >>>> opinion makes whole process a bit unclear/difficult (we don't
> > > >>>> have very good definitions for Alpha and Beta
> > > >>>
> > > >>> releases).
> > > >>>
> > > >>> Yes we do.
> > > >>>
> > > >>> Alpha means feature complete, asking for feedback on the API and
> > > >>> new functionality.
> > > >>>
> > > >>> Beta means implementation complete, asking for feedback on the
> > > >>> quality of the implementation, seeking bugs and regressions to
> > > >>> be fixed.
> > > >>>
> > > >>> RC means we've fixed everything we knew.
> > > >>
> > > >> I wholeheartedly agree with Thiago. The distinction may not make
> > > >> a
> > >
> > > difference from the release team perspective but it makes a world of
> > > difference for developers (who have different limitations for each
> > > step) and the world as they can decide how bleeding edge the code is
> > > that they want to test.
> > >
> > > >> I have a problem with the motivation behind this suggestion too
> > > >> (don't
> > >
> > > understand it). The naming makes no difference for you as release
> > > manager (afaict just a label). Why suggesting it aka what do you
> > > want to gain by doing snapshots only?
> > >
> > > >> --
> > > >> Alex
> > > >>
> > > >> _______________________________________________
> > > >> Development mailing list
> > > >> Development at qt-project.org
> > > >> http://lists.qt-project.org/mailman/listinfo/development
> > > >
> > > > _______________________________________________
> > > > Development mailing list
> > > > Development at qt-project.org
> > > > http://lists.qt-project.org/mailman/listinfo/development
> > >
> > > _______________________________________________
> > > Development mailing list
> > > Development at qt-project.org
> > > http://lists.qt-project.org/mailman/listinfo/development
> >
> > _______________________________________________
> > Development mailing list
> > Development at qt-project.org
> > http://lists.qt-project.org/mailman/listinfo/development
> 
> 
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development