[Development] Qt modules, API changes and Qt 6

[Frederik Gladhorn]

Hi all,

I'd like to start another discussion around our development workflow.
We arrived at our current model of Qt modules (in the git repository sense) 
and using qt5.git as a container for all of them through a series of steps and 
changes. Mix in the evolution of the testing environment over time and we have 
something that has grown in interesting ways.

I will try to describe the problem in this mail. I also have discussed with a 
bunch of people (inside The Qt Company) about potential solutions. After 
brainstorming and some back and forth, we had five suggestions on a whiteboard 
and picked our favorite. In a separate mail I'll try to describe these and 
what we concluded with as our favorite. All of this is up for discussion, so 
I'm hoping for someone to come up with even better ideas.


The problem:

qt5.git serves several purposes today, for example:
- it contains a few binaries needed for Windows development
- it gives a definition of which modules make up a Qt release
- it collects sets of revisions which work together (because they were tested 
with each other)
- it has a bit of build system infrastructure to build all of this

In my opinion there are a few issues with what we have:
- updating qt5.git (and thus making releases) is cumbersome
- it's unclear for many people what a module is tested against in the CI
  (its dependencies are checked out at the revisions specified in qt5.git, not 
the latest revisions of the corresponding branch)
- we have more products, such as Qt for Automotive, Automation, Device 
Creation, should those have qt5-special-purpose repositories?
- modules outside of qt5.git get different treatment, making it hard to 
include new modules

In the light of Qt 6, there is also the question of how we can allow making 
source incompatible changes in an easy fashion.
Why is it hard to do source incompatible changes? Let's say we want to rename 
a function in qtbase. qtdeclarative is the only user of that function in our 
code-base. Today we can do that in five easy steps:

1) add function with new name in qtbase, do not remove the old one yet
2) update qt5.git
3+4) move qtdeclarative to the new function name; remove the old function from 
qtbase
5) after 3 and 4 are in, update qt5.git

Five commits, two of which are qt5.git updates, is a lot to ask for. And this 
is assuming that the contributor knows of this work-flow and does everything 
by the unwritten book. In the unhappy case, we do an attempt of changing 
qtbase directly, then we notice qt5.git doesn't update and we need to bring 
back the old function name first. Another aspect is that even after 5, if this 
happened to be a branch other than dev, we might face all of these problems 
while doing merges to the next branch again (been there, done that, and so has 
Liang).

This has led to a mess in how we create releases and how we test things. It 
creates a very hard and artificial line of what's part of the core product and 
what is outside and testing in the CI is also majorly influenced by this.
Modules are not self-contained (since the fine-grained dependencies are 
specified in qt5.git). We don't know how to cleanly handle dependencies for 
modules outside. We have trouble getting some of the releases out, since Qt 
for Device Creation follows slightly different routines than Qt for 
Application Development

On a semi-related note, running git-bisect on anything but qtbase is a 
science, hard enough to at least put me off.

Cheers,
Frederik