[Android-development] Testing on Android

[Frederik Gladhorn]

On fredag 22. juni 2018 18.57.24 CEST Aleix Pol wrote:
<snip>
> Hi Frederik,
> Cool stuff. Maybe you could point at how these tests are being run? It
> will make it easier to offer any feedback.

Right, so what we've done so far is to simply launch a nightly job, the last 
one here:
https://testresults.qt.io/coin/integration/qt/qt5/tasks/nightly_1530679883

You should be able to find them just by looking at https://testresults.qt.io/
coin/?project=qt%2Fqt5 and searching for Android, the jobs will be "status 
checks" for now.

> 
> Are you packaging every test into an apk?

Indeed, we use androiddeployqt, which is probably not optimal (just like 
BogDan also wrote).
Avoiding that somehow would certainly be interesting. But keep in mind that we 
want to test something close to what our customers use. So we should test 
androiddeployqt, but presumably not with every test. Ideas how to implement 
something alternative (not copying the Qt libs over and over) would be 
appreciated. When I last asked around I was told Ministro is not working, I 
didn't spend time researching that option.

What we currently do:
launch the Android emulator
run make check with a testrunner script like this https://codereview.qt-project.org/#/c/232999/

There are some tricky bits: we don't get the exit code of the application/test 
being run, so we rely on the xml output. Qt used to produce invalid xml (due 
to being too generous with writing utf-8 or wrongly encoded data in data tags, 
xml is more restrictive).
As soon as we have that cleaned up (qtscript was the last culprit), we can 
start interpreting the testlib xml everywhere.

Right now the tests are red because of tests not even being launched. This 
will take some time to clean up and then we'll eventually also fix more and 
more tests I hope.

Cheers,
Frederik


> 
> Aleix