[Development] Allow to do parallel integration (was: Proposal: New branch model)

[Jedrzej Nowacki]

On Friday, January 25, 2019 11:08:52 AM CET Lars Knoll wrote:
> To me this means we need to seriously rethink that part of our CI system,
> and ideally test changes (or patch series) individually and in parallel. So
> maybe we should adjust our CI system that way:
> 
> * test changes (or patch series) individually
> * If they pass CI merge or cherry-pick them into the branch
> * reject only if the merge/cherry-pick gives conflicts
> 
> This adds a very small risk that two parallel changes don’t conflict during
> the merge/cherry-pick process, but cause a test regression together. To
> help with that, we can simply run a regular status check on the repo. If
> this happens the repo will be blocked for further testing until someone
> submits a fix or reverts an offending change, which is acceptable.
> 
> Another advantage is that this would pave the road towards a system where CI
> testing happens before human review, so we can in the longer term avoid
> duplicated review/approval work.

Hi,

 I do agree with it. Personally, I'm convinced that we are doing CI gate-
keeping in a wrong way. CI is about balancing costs vs. gains. We can not test 
everything, we should find as many as possible bugs, while running in 
reasonable time without using too much resources. That is the function to 
optimize.

 We really should test things in parallel. The risk of conflicts / regression 
caused by parallel integration in project as big as qt is minimal. One can be 
even proactive and not stage changes while other similar ones are integrating. 
While broken repository state is bad, it is also easy to detect; nothing 
integrates because of one test, which is not known to be flaky 
(testrestults.qt.io/grafana...). 

Cheap (a.k.a reasonably fast to implement) proposal:
- Stage button would create a stage branch as currently (through cherry-pick 
on top of target branch)
- 3-5 minutes old stage branch would changed to be a build branche and all 
patches in it would be changed to "integrating" state. CI would start testing 
the build branch. At that time a new stage branch may be created. As opposite 
to now, multiple build branches may be tested in parallel.
- Once a build branch testing passed we cherry-pick / merges (depends what 
gerrit does) changes on top of the target branch, in case of conflicts we mark 
it as failed.

Advantages:
 - changes are tested in smaller batches => smaller chances to fail
 - we can use hardware in a more efficient way => more code integrated per hour
 - fairly easy to implement => we would not need to wait for it to happen 
years
- that would be a step forward in to direction of testing before reviewing

Disadvantages: 
- risk of breakages going through CI 
- someone work needs to be done to enable it

Cheers,
  Jędrek