[Development] Qt CI reliability

[Jędrzej Nowacki]

On Saturday 30 of April 2016 20:26:20 Sean Harmer wrote:
> Hi,
Hi,

> after yet another 5 hour wait just to be greeted with yet another random
> failure with no build logs I'm getting really tired of the poor reliability
> of the Qt CI system.
I'm sorry about that.
 
> https://codereview.qt-project.org/#/c/157590/
> 
> has been greeted with genuine failures, failures in qtdeclarative,
> qtxmlpatterns, multiple random failures in qt3d despite being a simple
> change which I suspect are due to issues on one or more CI nodes.

I scanned through the failures and it seems that you had a very bad luck. I 
know CI should not be about luck and therefore I'm really sorry about that. 

You tried to stage the change 7 times
1. Failed to compile qt3d (broken change https://codereview.qt-project.org/#/c/157593/)
2. Looks like a network connectivity failure, logs were not flushed as they 
should, so you can blame CI
3. Blame CI, we failed to acquire a free machine for 5h, I will look at that 
later
4. Failed to compile qt3d (broken change https://codereview.qt-project.org/#/c/157590/)
5. Failed to compile qt3d (broken change https://codereview.qt-project.org/#/c/157590/), same as 4
6. Looks like a network connectivity failure, logs were not flushed as they 
should, so you can blame CI, same as 2
7. Passed

I will ask IT about network, it seems that network interface was re-configured 
during CI run and DHCP assigned a different IP. It should not happen (TM)

Rule of thumb is: if logs show broken compilation it means: real problem, 
don't blame CI. There are three main reasons, that I'm aware of, that can 
cause the problem (sorted according to the probability):
1. One of changes being integrated broke the compilation
2. One of module dependencies broke source compatibility
3. There was a untested template update (this reason will almost disappear 
soon)
*. There was huge radiation in Finland, but that you would know from the news 
;-)

Cheers,
 Jędrek