[Development] Maintainer action requested on tests

Ville Voutilainen ville.voutilainen at gmail.com
Tue Mar 27 10:53:08 CEST 2018


As many of you know, we've been operating a task force to reduce autotest
flakiness and improve the stability of COIN and related CI infra. This
has resulted in fair amounts of newly-blacklisted tests. What we need
maintainers
of various modules to do is as follows:

1) look at the blacklisted tests and see whether anything could be done
to un-blacklist them, aka fix the root cause

2) improve tests in general to reduce remaining flakiness. Grafana is your
friend to see which tests are flaky:
https://testresults.qt.io/grafana/d/000000012/overview?orgId=1

Please be careful about un-blacklisting; if you have trouble reproducing flaky
results or test failures, avoid the urge to blacklist just because
something works on
your machine; that doesn't help if the tests don't work in COIN. Talk to
Sami Nurmenniemi, Kari Oikarinen or me if you need help with
reproduction recipes etc.

Thanks in advance,
-me



More information about the Development mailing list