[Development] Flaky auto tests on macOS 10.12

[Morten Sørvig]

> On 19 Jun 2017, at 13:09, Tony Sarajärvi <tony.sarajarvi at qt.io> wrote:
> 
> It is the intention that we do this. But we're constantly running, not beside our horse, but after it. We have just begun activating ourselves to discuss about 5.10 platforms and we have FF coming up in two weeks (counting out July here).
> It took months to blacklist all the cases for 10.12 alone. So yes, I agree that how we do this is not good, but at some point we find it good enough. I ran qt5 builds through twice without failures and deemed it fit. Before that I kept blacklisting everything I saw (which actually isn't good either).
> 
> Dev branch is actually the place where they should be put yes. Unfortunately we are too late for that. We need 10.12 in 5.9, and for it to still be in dev branch, we should have put it in January. We were still working with our 5.8 platforms at that time. As I said...the horse is running away from us 😉


The process could be (using the current branches and os versions as examples):

- Add 10.12 to dev.
- Stabilize/fix/blacklist (backport changes to 5.9)
- Qt is now tested on 10.12. Also the 5.9 codebase, since it is similar to dev.

- Then add 10.12 to 5.9
- With less disruption since Qt already has been stabilized.

Morten