[Development] Blacklisting: CI vs non-CI
thiago.macieira at intel.com
Fri May 26 17:31:39 CEST 2017
On Friday, 26 May 2017 03:12:30 PDT Konstantin Tokarev wrote:
> If problem is that tests just don't have enough time to complete in CI
> environment, it would be a good idea to mark such tests as "slow" so they
> get higher timeout settings.
The problem is not the full test timeout. It's intermediate timeouts: every
time you use QTRY_VERIFY, QTRY_COMPARE, QElapsedTimer, QDeadlineTimer,
QTestEventLoop::timeout, etc., your test is now "flaky" because what you
measured to take a few microseconds can now take up to a second to run.
I've already decided I won't fix such issues. The CI needs to be brought up to
normal performance instead. That means we need to blacklist those tests for a
while. I'm simply asking how to hide those BPASS.
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
More information about the Development