[Development] Usage of forcesuccess and qt.tests.insignificant in CI branches

Sergio Ahumada sergio.ahumada at digia.com
Mon Dec 17 14:22:09 CET 2012

On 12/17/2012 01:17 PM, Anttila Janne wrote:
> Hi,
> I had some concerns related to change that was introduced lately to
> CI testconfigs:
>      https://codereview.qt-project.org/42954
>      https://codereview.qt-project.org/42955
> In short the question is whether we should have forcesuccess and
> qt.tests.insignificant configured on CI branch basis (dev, stable, release)
> or on CI repo basis?
> Sergio wants them on branch basis, and I can understand reasons/rationales
> he made. As far as I see the main benefit in this approach is that it
> is easy to remove these properties branch by branch, and make sure CI in
> any branch won't be blocked due to "incorrect" testconfig configuration.

Just to be clear.

I don't want them anywhere. That's why I am removing them.

insignificant tests should be marked as per test basis, if needed.

But back to the original email:

- On branch basis:

If there is a branch passing and two failing, we have at least one 
branch blocking on regressions.

- On repo basis:

If there is a branch passing and two failing, we don't do anything and 
we might be allowing regressions in the branch that was passing before 
*until* all of them are passing.

In any case, I think this is too much implementation detail of the CI 
system. The CI team should decide how to do it in order to increase the 
quality of Qt by catching regressions.

Sergio Ahumada
Release Engineer - Digia, Qt

More information about the Development mailing list