[Releasing] rethinking the branching scheme

Wed Feb 19 15:19:08 CET 2014

Hi,

as one of the people involved I agree that the current system is pretty 
broken. But I think we need to solve several issues and should attempt to do 
it incrementally.

Discussion points:

1. defining which patches are the branching points
2. awareness of branches (is it needed? we hope not, that's why we have dev 
and stable branches)
3. reluctance/inability to create patch releases
4. ease of contributing to Qt for casual developers
5. problems when merging branches
6. branch model and CI configuration

While I initially was very much for Ossi's proposal, I'd like to explore a few 
points for and against it. Maybe we can end up with something less radical.

Here are my thoughts to the issues listed above:

1. See my sha1 proposal mail.

2. This is a mixed thing. Ideally everyone knows to push to dev/stable. 
Pushing to the release branch is the exeption and can in my opinion be much 
harder as it should really not happen usually.
If a fix for a release is needed, we can demand that the person pushing the 
fix knows more (for example a branch name).

3. My impression is that we don't release enough patch releases. The reason is 
probably not simple. I suspect that we make the whole harder by not having 
branches, so we don't even see if a release is worthwhile and we don't 
encourage patches that would go into 5.1.x since there is basically now way to 
contribute them. This is also problematic for security fixes.
On a technical level it's trivial to create a branch from any tag/sha1, but it 
seems to be a more social problem to me.
Maybe having dev/stable/5.2/5.3/5.4 branches would make this easier.
Other issues are of course that each release has a big cost in QA and 
packaging etc.

4. Pushing to either dev or stable is great. I like that most people don't 
have to think about where to push. Since we have time based releases this is 
hardly a problem, the patch will be out soon. Demanding from everyone to 
follow and pushing to the 5.x.y branch seems to make it much harder for casual 
contributions.

5. See Ossi's mail. It didn't work. Merging in branches in two directions with 
the amount of changes we have is just not very practical. We work around the 
problem with locking branches and doing fast forward merges from dev->stable 
for example, but this is a huge issue and would be improved by the branching 
system suggested. This is a real problem for the release team and whoever else 
gets involved. The good thing is that this is mostly people working at Digia 
and does not effect people contributing on their spare time. I could still 
imagine spending our (my) time in a more productive way.

6. We support different configs/platforms from release to release. Our current 
infrastructure is set up so that we can have individual configs per branch. 
Changing how this is done also creates quite some re-organisation of how the 
CI is done, I am not sure how much work this would be.

Just some random collection of quick thoughts ;)

Greetings,
Frederik

Onsdag 19. februar 2014 12.47.01 skrev Oswald Buddenhagen:
> moin,
> 
> you may remember that we arrived at the dev/stable/release scheme after
> a some lengthly discussion a few years back.
> 
> now i'll explain that imo that scheme failed, and that we need to go
> back to a more traditional one. hurray!
> 
> the crucial issue is - surprise surprise - the CI system:
> - a straight downmerge just doesn't work due to the reverse
>   dependencies.
>   this can be temporarily disabled, but that is a hassle. also, even
>   when we tried that, things were a huge mess.
> - the workaround is doing a forward merge and then direct-pushing a
>   fast-forward downwards.
>   for this to work, the target branch needs to be locked down for a day
>   or two. that alone is obviously quite a disruption for people not
>   involved in the release process.
>   another lesson from today's experience is that despite fairly heavy
>   restrictions as to who can stage, we *still* got three "rogue" commits
>   in qtbase/stable today from people who happened to have the rights,
>   but were not involved in the release process this time. locking this
>   down even further, tailored to the particular situation each time
>   around, would be hassle (and thus error-prone).
> 
> so it basically comes down to non-atomicity, exacerbated by the enormous
> CI delay.
> the answer to that is quite obviously using an operation that is
> naturally atomic: branch creation.
> 
> on top of that, we already realized that we need the old/ branch
> namespace to be able to release from older versions (in case of security
> fixes). this is quite a hassle to maintain as well, and the asymmetry
> makes things hard to understand (and virtually impossible to actually
> release).
> 
> so i'm proposing that we switch to a master/5.x/5.x.y scheme as we had
> before opengov (and as we still have for qt creator).
> 
> the implications are, afaict:
> - we solve the downmerge problem ... by not having it in the first
>   place. only forward merges and branch creations.
>   the biggest advantage here is that branching can be done very quickly
>   in a uniform process by somebody from the release team, without
>   coordinating every step with half a dozen people.
> - CI configs will need to be cloned for each new branch. we need to make
>   sure that this is reasonably low-hassle.
> - one of the strong arguments for the current scheme was the purported
>   simplicity for the developers.
>   i think experience shows that this didn't really work out:
>   - the branches still have phases (e.g., "soft freeze" right after a
>     downmerge)
>   - people think in release versions anyway
>   - the "missed the deadline and need to cherry-pick" scenario continued
>     to exist, and was actually made much worse due to the fuzziness of
>     the date (again the CI delays). that's why we now have the staging
>     lockdowns on the *source* branches (i.e. dev around the dev =>
>     stable downmerge).
>   with the traditional scheme:
>   - people will need to figure out what "stable" is. no biggie - they
>     really do that anyway (in the other direction).
>   - pushing to a too high branch will still be prevented by the
>     staging lockdown for around two days
>   - pushing to a too low branch is no big deal, as we'll just forward
>     merge. at some point we'll lock down old branches, too (we
>     actually did that in the pre-opengov times)
>   - making anything but master the default git branch will be
>     unrealistic (we can't script updating gitorious and github). i don't
>     think that is a big deal.
> - i can retroactively create/move the branches for previous releases
> - we need a new picture for http://qt-project.org/wiki/Branch-Guidelines ;)
> - ...?
> 
> everybody on irc involved in the current disaster^Wrelease was in favor
> of giving this some serious consideration.
> 
> regards
> _______________________________________________
> Releasing mailing list
> Releasing at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/releasing

-- 
Best regards,
Frederik Gladhorn
Senior Software Engineer - Digia, Qt
Visit us on: http://qt.digia.com