[Development] Tests, Shadow-Build and Cross-Compilation

[Rohan McGovern]

Olivier Goffart said:
> On Sunday 29 January 2012 21:13:25 Holger Hans Peter Freyther wrote:
> > Hi all,
> > 
> > I finally got around to play with my personal jenkins[1] setup again,
> > specially to learn (and be kind of an example on how a non Tier1 platform
> > could ever become one).
> > 
> > There are some issues and I would like to discuss how these can be solved in
> > a way that doesn't look like an ad-hoc solution and that might scale.
> > 
> > I will mumble about the Problems, the Ad-Hoc solutions and muse about a
> > fix...
> > 
> > 
> > 
> > Problems:
> >   Shadow builds:
> > 
> > I always liked shadow builds as I could have several Qt configurations but a
> > command/shared source directory (e.g. X11, QWS, for i386 and ARM). It is
> > quite common for tests to break in shadow builds (the QA page in the wiki
> > discourages this as well).
> 
> Yes, the wiki recommands not to use shadowbuild because it often do not work.
> (I had the problem today again while trying to run qlibrary autotest)
> 

Hopefully this advice can be removed at some point.  There are 1-2
people gradually working on fixing up all the tests to work for shadow
builds, at which point it will be blocking in the codereview CI.
About half the modules (the simpler ones) are already set up for shadow
builds, they can be seen at http://wiki.qt-project.org/Public_Autotest_Infrastructure#linux-g.2B.2B_shadow-build_Ubuntu_11.10_x86

> [...]
> > 
> > Musing:
> > So somehow cross-compilation and shadow-builds suffer from the same problem
> > that accessing files from the source directory with a relative path. One
> > proposal would be to use something like 'QDir::addSearchPath("testdata",
> > SRCDIR)'" and have it set to the path of the .pro file, and access
> > everything using the prefix. Danimo mentioned that this will fail for
> > multimedia that might not use the Qt resource system. So we might want to
> > have a method in the testlib to resolve a path like "testdata:foo" to a
> > proper path? For cross compiling one could change the testdata prefix with
> > a command line option or environment variable?
> > 
> > The other part/wish would be to always have deployment targets for the
> > testcases and generate a run script or such as part of the installation.
> 
> There is QFINDTESTDATA and QTest::qFindTestData that have been added recently,  
> I think for this reason.  Have you heard of them?
> 

Yes they were added exactly for this.
There is also TESTDATA in .pro files to install test data files, but this is
not really good enough to be documented other than a comment in
testcase.prf.