[Development] The qtbase CI should run the qtdeclarative tests

[Rohan McGovern]

marius.storm-olsen at nokia.com said:
> While it's not normal to do this type of dependency "lock" for upwards dependencies, i think in this case we should indeed do the extra checking. Not just because QtDeclarative is such an important tech for Qt5, but also because it's a very good test case for QtBase.
> 

What would be the proposed workflow to resolve this situation?

  - I have a change in qtbase which I can't push because it breaks a
    qtdeclarative test.

  - I have a fix to the test in qtdeclarative but I can't push it
    because it doesn't pass without my qtbase change.

One solution I can think of is to introduce a %reverse_dependencies
in sync.profile, used for CI and nothing else, which can control the
SHA1 of qtdeclarative used while testing qtbase.

Another is to skip the testing of qtdeclarative in qtbase
<branchname>'s CI if qtdeclarative's sync.profile does not contain
"qtbase => refs/heads/<branchname>".

> The extra time used for checking both repos will give much greater stability as a whole.
> 

Well, we don't really know that.  Note that introducing qtdeclarative
into the qtbase tests means that both directions of breakage are now
possible (i.e. not only can qtbase changes block qtdeclarative's CI,
but now qtdeclarative changes will be able to block qtbase's CI - though
it should be unlikely).

However it's certainly reasonable to expect that, and give it a try.

> Maybe the fast platforms can do the extra checking, while slow platforms still handle only QtBase? At least we will capture most compile issues?

I think we'd start with adding new configurations rather than modifying
existing ones.  e.g. as there is currently a "linux-g++-32 Ubuntu 10.04
x86" configuration for qtbase, there could be a new "linux-g++-32 Ubuntu
10.04 x86 with declarative" configuration.

Doing it that way also means the "with declarative" configuration can
run just the qtdeclarative autotests (we assert that there's no need to
run the qtbase autotests again), theoretically not increasing the
overall runtime.

Unfortunately there's probably not enough resources to do that for some
platforms (like Mac).

> (The CI rules might get very complicated though.)

Yes, in some ways I feel this is adding complexity to the test setup to
work around a "false simplicity" in our source code setup.  We claim
that Qt is modular, but actually we know some parts of it are not really,
so we add gates to enforce some level of de-modularization.

> 
> Rohan/Sergio, what will be the turn-around increase if we always test Declarative together with QtBase?
> 

The runtime should be somewhere between the current runtime of qtbase,
and the qtbase runtime plus the qtdeclarative runtime.  Anything more
would be speculation ... but I don't think the runtime would be a big
issue :)

Anyway, overall I think it's worth a try, but we should accept that this
could also create some new problems as well as solving some existing
ones.