[Development] Removal of some of the blacklisted (non-working) autotests?

[Jedrzej Nowacki]

As you wrote, in the first iteration of the functionality I would go for an easy solution and I would detect just all modifications to tests/auto/* On the other hand I do not want to have the definition of stress testing in Coin code. Every one should be able to run it therefore it should be a make target.  So everyone could just type:
  make stress-test
in a test folder and after 10 min get the result.

Cheers,
  Jędrek
________________________________________
From: Edward Welbourne
Sent: Friday, November 4, 2016 9:36 AM
To: Jedrzej Nowacki
Cc: development at qt-project.org
Subject: Re: [Development] Removal of some of the blacklisted (non-working) autotests?

Jędrzej Nowacki contributed:
> Even more, we need something that would not allow flaky, badly written
> test to be merged to Qt in the first place, otherwise we would have
> that discussion again in next 12 months.

How practical would it be to coax CI/Gerrit into recognising when a
commit adds new tests, so as to include stress-testing of the new test
as part of the CI testing ?  A half-way-decent start would be to notice
any tests/auto/* files modified by the commit and have the testing make
check in the affected directories several times, instead of just once.
For bonus marks, when the test branch has several commits in it, only
fail the commits that do touch those test files, if those tests show a
mix of passes and fails.

        Eddy.