[Development] atomic commits across submodules

[Adam Treat]

On 12/18/2017 05:24 AM, Jędrzej Nowacki wrote:
> On czwartek, 7 grudnia 2017 12:17:16 CET Adam Treat wrote:
>> Hi,
>>
>> I think it is high time that we fix the underlying problem: supporting
>> atomic commits across submodules.
> As I'm not against the idea, I'm not really fan of it either. The problem with
> atomic commits across submodules is that they encourage API breakages. Without
> supporting both code paths even for a little while, one is not aware of the
> pain that our users needs to go through every time we break API.

So your position is we shouldn't need atomic commits across submodules 
because the changes that are leading to breakage are themselves the 
problem. Let's examine types of changes:

1) Build system changes that require cross-module modifications
2) New public API in base module where dependent module takes advantage 
of it
3) Private API being changed out from under
4) Merges of the above 3 types from release branches to dev

Of these, only #2 should be solvable without atomic commits. In that 
case, we can introduce API first and when it is integrated, only then 
update dependent modules. This is still a problem for #4 though, but 
more about that later.

For the other types, atomic commits are at least sometimes going to be 
required. Right now, we are experiencing breakage. And it *is* painful. 
But I have not seen a postmortem of said breakages that shows the 
current processes are up to preventing such breakage. Maybe the answer 
is to move away from private API and a build system that does not need 
to be bootstrapped from Qt...

Now, back to #4, I do know that those responsible for merging from 
release branches into dev face a mind field of cross-module 
modifications. Consider #2 again and say such a change across modules 
has made it into release branches. It is my understanding that it is 
*just such a set of changes* that led to Liang Qi's request to add 
labels to API changes because the ordering info gets lost.  Merges from 
release to dev are going to be problematic until we have some way of 
ensuring cross-module dependencies are handled.

Which is why I propose that we stop merging from release to dev and 
require all changes to hit dev first. But, more on that in another email 
thread later. Maybe after the holidays ;)

>> Once this is done we should revert our CI to test changes against latest
>> version of all modules.
> No. First, how these two are connected, that is a big jump to a solution,
> without providing reasoning. I know that you didn't know about the fact that
> we pin submodules in qt5 and you got burned by this. That __is__ an argument,
> but an alternative solution would be a better documentation or UI.  Second,
> the most important, that doesn't scale. By compiling against latest Qt5 we
> reduced CI load by ~80%.

Fair enough, let's table this. I have more to say about this, but it 
really is related to my suggestion above. Again, more about that later.