[Development] Tests, Shadow-Build and Cross-Compilation

[Holger Hans Peter Freyther]

Hi all,

I finally got around to play with my personal jenkins[1] setup again, 
specially to learn (and be kind of an example on how a non Tier1 platform 
could ever become one).

There are some issues and I would like to discuss how these can be solved in a 
way that doesn't look like an ad-hoc solution and that might scale.

I will mumble about the Problems, the Ad-Hoc solutions and muse about a fix...



Problems:
  Shadow builds:

I always liked shadow builds as I could have several Qt configurations but a 
command/shared source directory (e.g. X11, QWS, for i386 and ARM). It is quite 
common for tests to break in shadow builds (the QA page in the wiki 
discourages this as well).

  Cross compilation:
When cross compiling the machine that compiles the tests will not be the 
machine that ends up running them.


Ad-hoc solutions:
  Shadow builds:

The ad-hoc solution appears to change tests that refer to files in a way like 
this:
	VERIFY(file.open("testfile"));

to:
	VERIFY(file.open(SRCDIR "testfile"));

and put DEFINES += SRCDIR=... into the pro file.


  Cross compilation:

The attribute of cross-compilation and running the test on a different 
architecture is somehow attributed to both Windows-CE (and Symbian). This 
manifests in the xmlpatterns test by adding deployment targes, deploy files, 
change the sourcecode to access different directories.


Musing:
So somehow cross-compilation and shadow-builds suffer from the same problem 
that accessing files from the source directory with a relative path. One 
proposal would be to use something like 'QDir::addSearchPath("testdata", 
SRCDIR)'" and have it set to the path of the .pro file, and access everything 
using the prefix. Danimo mentioned that this will fail for multimedia that 
might not use the Qt resource system. So we might want to have a method in the 
testlib to resolve a path like "testdata:foo" to a proper path? For cross 
compiling one could change the testdata prefix with a command line option or 
environment variable?

The other part/wish would be to always have deployment targets for the 
testcases and generate a run script or such as part of the installation.

comments
	holger



[1] http://qt-jenkins.moiji-mobile.com/jenkins/