[Development] Proposal: Eliminate soft branching phase from release process

Edward Welbourne edward.welbourne at qt.io
Tue Oct 15 12:37:56 CEST 2019

Kari Oikarinen (Friday, October 11, 2019 3:04 PM) wrote:
>>> I want to propose eliminating soft branching phase and instead use
>>> the creation of the branch as a cut-off for feature freeze (or bug
>>> fixes for a patch release). Frederik already alluded that there has
>>> been some discussion about making this change in the email about the
>>> final downmerge to 5.13.2.
>>> Right now a week before feature freeze the new branch is created and
>>> the release manager sends a heads-up mail that tells people to
>>> finish their changes within a week or retarget them to the new
>>> branch. Then after the week is up the original branch is merged into
>>> the new branch. This is called a "downmerge", since it is in the
>>> opposite direction compared to the usual merges that go "up" to
>>> newer releases.  Once the downmerge is done, branching is considered
>>> complete and feature freeze is in effect.
>>> Similar week of soft branching is also involved when patch level
>>> branches are created.
>>> As far as I can tell, the major motivation for providing this week
>>> of soft branching is to allow people to finish their last changes
>>> without needing to retarget the change to a new branch. Retargeting
>>> of a large number of changes would also have provided a large amount
>>> of work to Gerrit admins, since on our old Gerrit retargeting the
>>> change needed admin rights. And of course admin response time would
>>> have served as a bottleneck to work.
>>> Nowadays everyone can however move their own changes over to a new
>>> branch.

None the less, they then need to get re-approved, which can be a hassle.

Ossi provided two more reasons for it:
>> the reason is that the huge CI delay makes it impractical to know 
>> whether a change will still land in time for branch creation.

We need to distinguish two "CI delay" cases here: one is the per-attempt
delay (it's several hours); the other is the delay from getting +2 to
having one of those attempts succeed, which can take days.  Folk need at
least the former's warning of the point when they must switch branches
and life would be easier if they had the latter's warning.  In
particular, when a change is approved before feature-freeze but Coin has
been flaking on it repeatedly (without it being implicated as the cause
of that flakiness), do we really want to hold it out of the release ?

>> also, people just don't pay attention to the branching announcements
>> in real time.

That can be solved by competent communication - general announcement of
the schedule a week in advance, final warning about a day before, clear
announcement as soon as the deed is done.  Those targetting the new
branch then know to hold off on staging for the last Coin-attempt cycle
before branching.

>> so the alternatives to soft branching are

>> a) restricting the source branch before branching (which is obviously
>> counterproductive) or

If the duration is short, the counterproductivity is limited.  We only
really need to restrict for one Coin-attempt (albeit a conservative
estimate at the slowest attempt).

>> b) accepting the need for a significant number of cherry-picks (which
>> are a hassle to make, and an annoyance in the resulting git history).

Do we have statistics on how many commits Coin typically integrates to
dev in a day ?  (Ideally from the weeks around an actual branching.)  I
don't like cherry-picks, but if the numbers are modest we can live with

Back to Kari:
>>> So as our tools have improved, I see a chance to simplify our
>>> processes as well.
>>> So instead of the soft branching process I propose the following:
>>> - A week before feature freeze date release manager sends a reminder
>>>   email about it. This provides the same useful warning as the
>>>   initiation of soft branching has so far.

We should do this anyway !  (... and I imagine we do.)

>>> - On feature freeze day release team creates the new branch. They send
>>>   an email to development list informing people about the feature
>>>   freeze being in effect.

Send a mail a Coin-cycle before branching to say it's imminent.
As soon as the branch exists, announce its existence.

>>> - If your change did not make it into dev before the new branch was
>>>   created, it did not make the feature freeze cut.

This is problematic when the reason it didn't get through was that Coin
was flaking out, which it does all too often.  That can be addressed by
making (well-defined, limited) provision for a change that was already
approved before branching to be allowed to move onto the new branch to
get integration sorted out, if the only problem is Coin flaking out.  We
would need to think carefully about the rules for that, or feature
freeze could end up slushy.

>>>   If it is a bug fix that should go to the next release still, you
>>>   need to move it to the new branch. If the change happened to be
>>>   integrated after branch creation but before you noticed, you need
>>>   to cherry-pick it to the new branch. But that should hopefully be
>>>   an exception.

I would like statistics on how many integrations move through Coin per
day before taking for granted that this exception is rare enough; but it
does sound plausible.

>>> - There are no more downmerges. All merges happen in the same
>>>   direction. Hopefully that makes how they happen easier to
>>>   understand.
>>> The same approach should be used for patch level branches as well.

The patch-level case doesn't have a feature freeze tied to it, which
makes everything easier and the case for down-merge weaker.

>>> Currently there is a bit of uncertainty about when exactly the
>>> downmerges happen,

I confess I seldom find this a problem at present.  If my change belongs
on the new branch, I've had a week to move it before the down-merge, so
the uncertainly isn't a big deal.  If my change *shouldn't* go to the
new branch, I just wait for the announcement that soft-merging is done.
No problem.

>>> since it requires not only someone to do it, but also CI not being
>>> active in the target branch. That's a requirement because downmerges
>>> (if they have no conflicts) are pushed directly to the repositories
>>> instead of going through regular CI. Eliminating them also
>>> eliminates this irregularity. It hasn't been a big problem, but
>>> these commits have no corresponding Gerrit changes, which has
>>> confused people when they can't actually find the review because
>>> there wasn't one. It can also result in broken state of code,
>>> although only rarely.  By getting rid of downmerges we eliminate the
>>> vast majority of direct pushes (and all regularly done ones).

It would indeed be nice to see the last of routine direct pushing.

>>> Since branches can be created without waiting for idle CI, the timing
>>> of feature freeze coming into effect could become better known in
>>> advance. This helps in avoiding confusion about whether it's in effect
>>> or not.

What problems has such confusion caused in the past ?

>>> If this is approved, I promise to edit QUIP-11 [1] to match.
>>> [1]: https://codereview.qt-project.org/c/meta/quips/+/228450 -
>>>      Hopefully it will be merged at some point...

It's due an update, anyway - see existing review comments.
Getting it merged before that update isn't going to happen,


More information about the Development mailing list