[Development] Need advise on acceptable timeouts for autotests

[Marc Mutz]

Hi,

We repeatedly have the problem that timeouts that developers think are ample 
(because they exceed typical runtime by, say, two orders of magnitude) are 
found to be insufficient on the CI.

Latest example: 
http://testresults.qt.io/coin/integration/qt/qtbase/tasks/1489618366

The timeout to run update-mime-database was recently increased to 2mins. But 
that still does not seem to be enough. For a call that hardly takes a second 
to run on unloaded machines.

We can of course crank up timeouts to insane amounts like 1h, but that means 
that a test will just sit there idling for an hour in the worst case.

I have two questions:

1. Where do these huge slowdowns come from? Is the VM live-migrated? Is the
   SAN, if any, down? At this point it lools like no overcommitting of CPU/RAM
   could ever explain how update-mime-database can take 2mins to run.

2. What should we choose as timeouts? I understand that tests which are stuck
   are killed after some time (how long?). Maybe timeouts should be set to the
   same value?

Thanks,
Marc

-- 
Marc Mutz <marc.mutz at kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts