[Qt-interest] regression testing Qt

[Pavel Heimlich, a.k.a. hajma]

Hi,
I am one of the guys porting KDE4 to Open/Solaris. Our Qt build uses a
couple of hacks/patches and is built with Sun Studio with a lot of
custom build flags.

I'd like to regression test our build vs. a clean build and if I find
discrepancies I'd like to run the tests for each build flag and patch.

So far I haven't found (m)any references of people using the autotests
in a more sophisticated way (not much apart
http://qt.gitorious.org/qt/pages/QtAutotestsEnvironment,
http://qt.gitorious.org/+kde-developers/qt/kde-qt/blobs/README/README.kde-qt).

I am slightly confused about how to run the tests.
The test.pl script worked fine, but at
http://labs.trolltech.com/blogs/2009/11/05/analyzing-benchmark-results-with-bm/,
Rohan McGovern says 'AFAIK tests/auto/test.pl is not used by anyone'
Running them via 'make install' went well too, but when a particular
test stuck, there was no timeout-and-kill, so the whole thing just
stuck. Also there was no resuls files produced that way (but I could
just redirect it to a text file).
The Qt guys on IRC confirmed me that they use some QProcess app to run
the tests internally. Also that a replacement target for running the
checks would be 'make check' with some more features, but that will
only be in the future.

Would BM http://labs.trolltech.com/blogs/2009/11/05/analyzing-benchmark-results-with-bm/
be a suitable tool? ("bmrunner, a simple console-based tool that
simply runs all QTestLib-based benchmark programs it can find in a
directory subtree and uploads the results to the database.")
Has anyone tried it out?

Or would I spend more time by making it work than by putting together
a shell script to handle the output of test.pl ?

So far I managed to build all (all but two in fact) the tests so I'm
only at the beginning.

Any hints appreciated.

hajma