[Development] Running Qt auto-tests offscreen

[Samuel Rødal]

Hello,

do you use the CI system as a way to run the auto-tests? Are you tired 
of having to wait forever for the widgets auto-tests to run, at the same 
time as they pop up a hundred windows and dialogs that steal the focus 
and prevent you from doing any work? In that case I'm working on a 
solution for you.

I've introduced a platform plugin called "offscreen", primarily intended 
for testing purposes. It mocks up all the platform integration classes 
with the minimum functionality required for the tests to run and pass, 
without actually showing anything in the windowing system. That means 
that all the QtWidgets tests or QtQuick tests for instance can be run in 
parallel.

On my i7-2600k, running the QtWidgets tests in sequence with the xcb 
plugin takes 8m20.244s as reported by the "time" command, whereas 
running them with the offscreen plugin takes only 1m56.746s, and it's 
possible to actually use the system while they run. A lot of the tests 
also contain unnecessary qWait() commands that could be reduced, 
removed, or replaced by QTRY_VERIFY or similar to further reduce the 
running times.

The plugin isn't yet fully at par with xcb when it comes to the number 
of failing tests, but it should be possible to get there. If it weren't 
for OpenGL, the plugin could probably be completely platform independent 
(apart from needing a trivial event dispatcher similar to 
QUnixEventDispatcherQPA, and assuming we use a platform-independent font 
database such as QBasicFontDatabase). Currently it only works on Linux 
(with GLX if you want OpenGL, by creating an X window that's never 
mapped and using glXMakeCurrent etc on it). Similar functionality should 
be possible to implement on Windows and Mac.

In the end, maybe we could even use this for a subset of the tests in 
the CI system. In my opinion, with the current platform abstraction 
layer we have in place in QtGui, it's unnecessary for most of the 
widgets tests to actually test the entire windowing system stack. 
QWidget is now implemented on top of QWindow and QBackingStore in QtGui, 
so most of the testing of the platform plugins and windowing system 
integration instead of the actual Qt code should be done there.

Here's the change: https://codereview.qt-project.org/#change,41858

Btw, networking tests and similar that consume system resources will 
still not be able to run in parallel.

--
Samuel