[Development] Proposal: New branch model

[Konstantin Tokarev]

23.01.2019, 18:54, "Jedrzej Nowacki" <jedrzej.nowacki at qt.io>:
> Hi,
>
>   It is time to rethink our branch model. We are approaching Qt6 development
> and I'm worried that what we have now, will simply not scale. As you know, our
> branch model is mainly(*) based on merging from stable up to development
> branches. In general, it is a very good model, which make sure that release
> branches are not getting obsolete too quickly, that most of the fixes are in
> the right places, that every commit is only once in the git history. It is a
> very clean model. It is also a very slow model, with a single point of
> failure.
>
>   It is hard to maintain:
>   My impression is that the current model works great for a single release
> branch, but we have: 5.6 5.9 5.12 and soon we will have 5.13, that is without
> counting temporary patch level branches. Merging through them is hard, we
> already had to use "cherry-pick mode" in some places. When we add to the
> picture dev and qt6 branches, merging will be very, very hard. It is
> practically a full time job to update qt5 repository and coordinate all the
> merges now (thank you Liang!), shortly after qt6 branch opening amount of work
> will be much bigger.

Note that backporting changes from dev should also be a full-time job for someone,
otherwise amount of fixes going to stable branches will likely drop

>   It is slow:
>   The merges take time. We produce a lot of code, we have a lot of tests that
> needs to pass. Even single failure delays merge propagation by at least one
> day. If by bad luck, the merge contains some API incompatible changes an
> intermediate jump through Qt5 integration is required, that adds at least 3
> days of delay.
>
>   ------------------------------------------------------------------
>
>   Proposal in short: let's use cherry-pick mode everywhere.
>
>   All(**) changes would go to dev. From which they would be automatically
> cherry-picked by a bot to other branches. The decision to which branch cherry-
> pick, would be taken based on a tag in the commit message. We could add a
> footer that marks the change risk level as in quip-5 (http://quips-qt-io.herokuapp.com/quip-0005.html), so for example "dev", "stable", "LTS". By
> default everything would be cherry-picked to qt6 branch unless "no-future" tag
> would be given. Of course we can bike-shed about the tag names.
>
>   Advantages:
>   - no waiting for merges, a fix can be used right away after integration
>   - faster propagation of fixes targeting all branches, as there are no merges
> of merges
>   - simpler for new contributors, always push to dev
>   - in most cases a reviewer does no need to know what the current version
> number is
>   - conflict resolution is distributed and affects mostly the author of the
> change
>   - documents a change intent, which may be useful for people keeping own
> forks
>   - over time with increased amount of conflicts old branches, in natural way,
> stay untouched
>
>   Disadvantages:
>   - git history would be a bit wilder, "git branch --contains" would not work
>   - commit messages in some branches would have kind of ugly footer as an
> effect of "cherry-pick -x"

Gerrit's Change-Id can be used to track presence of patch in branches of interest

>   - there is a chance, that some cherry-picked commits may be left forgotten
> in gerrit after a failed integration

Or not submitted at all because of failed cherry-pick. Also, notorious property of 
cherry-pick strategy is explosion of conflicts: if there is a patch series modifying
some code, and first of them requires adjustment to resolve conflict, it's likely that
all following patches will have conflicts as well in the same places.

>
>   Bot details:
>
>   The bot would listen only for changes in dev, in some unusual cases one
> could target an other branch directly, but bot would not trigger automatic
> cherry-pick(***). The bot would wait for a successful dev integration before
> creating cherry-picked changes. The bot would use cherry-pick -x to annotate
> the origin patch. After the cherry-pick creation, it would push it to gerrit,
> +2 and stage once. It would be up to the author to re-stage in case of
> flakiness. In case of a cherry-pick conflict it should push unresolved conflict
> to gerrit and add all reviewers and author to handle the issue.
>
> The list below shows branch targets for automatic cherry-pick based on given
> tag.
>
> dev (qt6)
> stable (qt6, 5.13)
> stable, no-future (5.13)
> LTS (qt6, 5.13, 5.12)
> LTS-strict (qt6, 5.13, 5.12, 5.9)
> LTS, no-future (5.13, 5.12)
>
> That is assuming that we have branches: qt6 dev 5.13 5.13.q 5.12 5.12.w
> 5.9 5.9.e 5.6 5.6.r
> I think we should assume that patch level branches, as well LTS in very strict
> state, are handled manually.
>
>   Creation of new branches:
> We would branch more or less as usual. The only difference from the current
> system is that we would not need the back merge / soft branching anymore, but
> of course we could keep it.
>
> --------------------------------------------------------------------
>
>   Why not X instead?
>   ------------------
>
>   - GitFlow, GitHub <= both are based on feature branches, that doesn't work
> well with gerrit.
>   - Stay with the current solution <= the merge effort is too big and qt6 is
> expected to cause conflicts that really should not be solved by one person
>
>   What to do around Qt6 release?
>   ------------------------------
>
>   Replace dev branch with qt6 branch content, do not use "no-future" tag
> anymore. From a random contributor perspective nothing changes.
>
>   Can we use annotate instead of cherry-pick -x?
>   ----------------------------------------------
>
>   No, but we should use it in addition. Annotations are mutable, therefore are
> not 100% trust worthy. If we have both, we could validate annotations against
> commit message, sadly it would not protect against annotation removal.
>
>   Can we use gerrit change id instead of cherry-pick -x?
>   ----------------------------------------------
>
>   Yes, but it would tie us more with gerrit. I do not think it is good in long
> term, but as long we are using gerrit one can use gerrit id as a unique id of
> a concept behind a change (a change my look differently in different branches).
>
>   How migration would look like?
>   ------------------------------
>
>   It would be quite automatic. We need to:
>   1. block access to gerrit
>   2. re-target all changes to dev branch (a part from topic branches) and
> amend commit message based on the original target branch
>   3. enable warning in sanity bot for all branches, excluding dev and topic
> branches, so it warns about wrongly pushed changes
>   4. enable cherry-pick bot
>   5. unblock gerrit access
>
>   What about existing topic branches?
>   -----------------------------------
>
>   Nothing really. We still can use them and merge them to dev with a right
> tag.
>
>   What if we need a release blocker fix, right now!
>   -------------------------------------------------
>
>   The setup prioritize dev. So a release branch would get a fix with a delay of
> one dev integration. In __exceptional__ situations nothing prevent us from
> cherry-picking in the opposite direction.
>
> Cheers,
>   Jędrek
>
> (*) because of the issues described in the email, some LTS branches are
> already in the cherry-pick mode.
> (**) with exception of branch specific changes.
> (***) unless proven to be useful
>
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> https://lists.qt-project.org/listinfo/development

-- 
Regards,
Konstantin