[Releasing] rethinking the branching scheme

Wed Feb 19 17:35:05 CET 2014

On Feb 19, 2014, at 3:19 PM, Frederik Gladhorn <frederik.gladhorn at digia.com> wrote:

> Hi,
> 
> as one of the people involved I agree that the current system is pretty 
> broken. But I think we need to solve several issues and should attempt to do 
> it incrementally.
> 
> Discussion points:
> 
> 1. defining which patches are the branching points
> 2. awareness of branches (is it needed? we hope not, that's why we have dev 
> and stable branches)
> 3. reluctance/inability to create patch releases
> 4. ease of contributing to Qt for casual developers
> 5. problems when merging branches
> 6. branch model and CI configuration
> 
> While I initially was very much for Ossi's proposal, I'd like to explore a few 
> points for and against it. Maybe we can end up with something less radical.
> 
> Here are my thoughts to the issues listed above:
> 
> 1. See my sha1 proposal mail.
> 
> 2. This is a mixed thing. Ideally everyone knows to push to dev/stable. 
> Pushing to the release branch is the exeption and can in my opinion be much 
> harder as it should really not happen usually.
> If a fix for a release is needed, we can demand that the person pushing the 
> fix knows more (for example a branch name).

> 3. My impression is that we don't release enough patch releases. The reason is 
> probably not simple. I suspect that we make the whole harder by not having 
> branches, so we don't even see if a release is worthwhile and we don't 
> encourage patches that would go into 5.1.x since there is basically now way to 
> contribute them. This is also problematic for security fixes.
> On a technical level it's trivial to create a branch from any tag/sha1, but it 
> seems to be a more social problem to me.
> Maybe having dev/stable/5.2/5.3/5.4 branches would make this easier.
> Other issues are of course that each release has a big cost in QA and 
> packaging etc.

To create a release from a sha, you need to know the sha.
Current stable (up to the merge from dev) contains fixes for tasks that are now closed with fix version Qt 5.2.2. If we never do a 5.2.2 release that doesn’t really matter, but if we do a 5.2.2 release it should contain these fixes, e.g. it should be done from “current stable” (plus maybe some patches, who knows).
We have to “remember" that sha. A natural way to do that is to have it as HEAD of a 5.2 branch. I think if we have to start digging for what sha to release as a 5.2.2, we will for sure not do it. For as long as we do not use the release branch for alpha releases (actually why don’t we?), and didn’t do a beta release for the new minor release yet, the release  branch can be used to cache the “potential 5.2.2” (that’s still pretty non-obvious btw). But the moment the release branch is used for anything else, the sha is just some sha in some tree of trees...

> 4. Pushing to either dev or stable is great. I like that most people don't 
> have to think about where to push.

Pushing without thinking is evil.

>  Since we have time based releases this is 
> hardly a problem, the patch will be out soon. Demanding from everyone to 
> follow and pushing to the 5.x.y branch seems to make it much harder for casual 
> contributions.

Someone who doesn’t know our branch policy doesn’t know if he should push to dev or stable (or release) either.
IMO our policy with dev/stable/release is also not easier than it would be with a dev branch and 5.x branches. It *claims* to be easier, but in reality it isn’t. For example it obfuscates in what releasing phase of what version of Qt we currently actually are. *Especially* as a casual contributor to Qt you actually have no idea what you are checking out and submitting patches to, and I think that is not a good mode to operate in.

> 5. See Ossi's mail. It didn't work. Merging in branches in two directions with 
> the amount of changes we have is just not very practical. We work around the 
> problem with locking branches and doing fast forward merges from dev->stable 
> for example, but this is a huge issue and would be improved by the branching 
> system suggested. This is a real problem for the release team and whoever else 
> gets involved. The good thing is that this is mostly people working at Digia 
> and does not effect people contributing on their spare time. I could still 
> imagine spending our (my) time in a more productive way.
> 
> 6. We support different configs/platforms from release to release. Our current 
> infrastructure is set up so that we can have individual configs per branch. 
> Changing how this is done also creates quite some re-organisation of how the 
> CI is done, I am not sure how much work this would be.
> 
> Just some random collection of quick thoughts ;)
> 
> Greetings,
> Frederik
> 
> 
> Onsdag 19. februar 2014 12.47.01 skrev Oswald Buddenhagen:
>> moin,
>> 
>> you may remember that we arrived at the dev/stable/release scheme after
>> a some lengthly discussion a few years back.
>> 
>> now i'll explain that imo that scheme failed, and that we need to go
>> back to a more traditional one. hurray!
>> 
>> the crucial issue is - surprise surprise - the CI system:
>> - a straight downmerge just doesn't work due to the reverse
>>  dependencies.
>>  this can be temporarily disabled, but that is a hassle. also, even
>>  when we tried that, things were a huge mess.
>> - the workaround is doing a forward merge and then direct-pushing a
>>  fast-forward downwards.
>>  for this to work, the target branch needs to be locked down for a day
>>  or two. that alone is obviously quite a disruption for people not
>>  involved in the release process.
>>  another lesson from today's experience is that despite fairly heavy
>>  restrictions as to who can stage, we *still* got three "rogue" commits
>>  in qtbase/stable today from people who happened to have the rights,
>>  but were not involved in the release process this time. locking this
>>  down even further, tailored to the particular situation each time
>>  around, would be hassle (and thus error-prone).
>> 
>> so it basically comes down to non-atomicity, exacerbated by the enormous
>> CI delay.
>> the answer to that is quite obviously using an operation that is
>> naturally atomic: branch creation.
>> 
>> on top of that, we already realized that we need the old/ branch
>> namespace to be able to release from older versions (in case of security
>> fixes). this is quite a hassle to maintain as well, and the asymmetry
>> makes things hard to understand (and virtually impossible to actually
>> release).
>> 
>> so i'm proposing that we switch to a master/5.x/5.x.y scheme as we had
>> before opengov (and as we still have for qt creator).
>> 
>> the implications are, afaict:
>> - we solve the downmerge problem ... by not having it in the first
>>  place. only forward merges and branch creations.
>>  the biggest advantage here is that branching can be done very quickly
>>  in a uniform process by somebody from the release team, without
>>  coordinating every step with half a dozen people.
>> - CI configs will need to be cloned for each new branch. we need to make
>>  sure that this is reasonably low-hassle.
>> - one of the strong arguments for the current scheme was the purported
>>  simplicity for the developers.
>>  i think experience shows that this didn't really work out:
>>  - the branches still have phases (e.g., "soft freeze" right after a
>>    downmerge)
>>  - people think in release versions anyway
>>  - the "missed the deadline and need to cherry-pick" scenario continued
>>    to exist, and was actually made much worse due to the fuzziness of
>>    the date (again the CI delays). that's why we now have the staging
>>    lockdowns on the *source* branches (i.e. dev around the dev =>
>>    stable downmerge).
>>  with the traditional scheme:
>>  - people will need to figure out what "stable" is. no biggie - they
>>    really do that anyway (in the other direction).
>>  - pushing to a too high branch will still be prevented by the
>>    staging lockdown for around two days
>>  - pushing to a too low branch is no big deal, as we'll just forward
>>    merge. at some point we'll lock down old branches, too (we
>>    actually did that in the pre-opengov times)
>>  - making anything but master the default git branch will be
>>    unrealistic (we can't script updating gitorious and github). i don't
>>    think that is a big deal.
>> - i can retroactively create/move the branches for previous releases
>> - we need a new picture for http://qt-project.org/wiki/Branch-Guidelines ;)
>> - ...?
>> 
>> everybody on irc involved in the current disaster^Wrelease was in favor
>> of giving this some serious consideration.
>> 
>> regards
>> _______________________________________________
>> Releasing mailing list
>> Releasing at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/releasing
> 
> -- 
> Best regards,
> Frederik Gladhorn
> Senior Software Engineer - Digia, Qt
> Visit us on: http://qt.digia.com
> 
> _______________________________________________
> Releasing mailing list
> Releasing at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/releasing

-- 
Eike Ziller, Senior Software Engineer - Digia, Qt

Digia Germany GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B