[Development] BlackBerry Jenkins server

[Turunen Tuukka]

Looks very good.

In order to have BB10 Tier 1 one day reliable CI and fast build cycles is certainly needed. The more there are platforms to test, the more likely it is that something goes wrong causing harm to all platforms.

For the test results, we have some ideas how to produce these in more user friendly form. But for now jenkins results are good to have.

Yours,

Tuukka

________________________________
Lähettäjä: development-bounces+tuukka.turunen=digia.com at qt-project.org [development-bounces+tuukka.turunen=digia.com at qt-project.org] käyttäjän Wolfgang Bremer [wbremer at rim.com] puolesta
Lähetetty: 12. huhtikuuta 2013 18:17
Vastaanottaja: development at qt-project.org
Aihe: Re: [Development] BlackBerry Jenkins server

Hi there,

this is another update regarding the Blackberry CI server.

We are actually testing more Qt branches concurrently on multiple devices.
The system is working in this manner already since about a month and looks stable enough for a further upstream integration.

Where?
http://qtlab.blackberry.com/ is the official public website

Which Qt versions/branches?
The following branches are currently built and tested.
Qt-4.8, Qt-4.8-bb10
Qt-5 dev and stable (qtbase -> qtjsbackend -> qtxmlpatterns -> qtdeclarative)
(QtSensors, Qt3d, QtConnectivity, QtGraphicalEffects, QtLocation probably following soon)

How?
The mentioned branches are cloned, built and afterwards the auto tests executed on the attached devices.
The specific jobs are started periodically at fixed time slots.
At first the x86 architecture is built for the BB10 simulator but not tested.
It would be necessary to start multiple simulator instances to speed up the testing for x86.
If the x86 job was successful, the build procedure for ARM is started and afterwards tested.
At the moment there are 7 devices attached to the server, but we aim at 8 devices(different types) in total.

Which auto tests?
Currently some test sets are excluded because they are unstable or simply produce core dumps.
A list of blacklisted tests is available together with a reason separately for Qt4 and Qt5.
Qt4 blacklist: https://qt.gitorious.org/+rim-qt-developers/qt/rim-qtqa/blobs/master/scripts/blacklist-qt4-tests
Qt5 blacklist: https://qt.gitorious.org/+rim-qt-developers/qt/rim-qtqa/blobs/master/scripts/blacklist-qt5-tests

Test results?
I've chosen the xunit Jenkins plugin to analyze and present the testing results.
It is very good at presenting the results but sometimes fails to set the correct build result.
Probably this will need further investigations.
If you are interested in the test results of a specific Qt branch just do the following:
Choose a job e.g. qt-5-qtbase-stable, select the architecture ARM(x86 does only contain one fake result)
and finally "Latest Test Result".
This will give you an overview of all failed tests:
http://qtlab.blackberry.com/job/qt-5-qtbase-stable/ARCH=ARM/lastCompletedBuild/testReport/
If you want to know the trend of recent results click on "History":
http://qtlab.blackberry.com/job/qt-5-qtbase-stable/ARCH=ARM/lastCompletedBuild/testReport/history/?#

What's next?
We are looking forward to an upstream integration together with the qt-project.org CI system.
Could the responsible guys at Digia/Qt-Project please stand up? :)

On 01.02.2013 05:31, Peter Hartmann wrote:

On 11/07/2012 03:30 PM, Peter Hartmann wrote:


(...)
But our plan is absolutely to have test and build results public, and
make the setup stable enough to at some point tie into the Qt CI
infrastructure.


Update: the server is now online at http://195.3.174.130/ . Currently it
is only running builds for x86 and ARM (scripts at
https://qt.gitorious.org/+rim-qt-developers/qt/rim-qtqa), because we
cannot attach devices to run tests yet (company security policy etc.).

Please do not rely on it in any way (not even the IP), we will be
changing the setup the next days / weeks; this is just a notification
that there is progress :)

Peter

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
_______________________________________________
Development mailing list
Development at qt-project.org<mailto:Development at qt-project.org>
http://lists.qt-project.org/mailman/listinfo/development



--
Wolfgang Bremer

Software Engineer (CI & QA)
Qt Team

Office: 0049 (0)234 43669826 x42826

BlackBerry: 0049 (0)162 2478353

PIN: 25BF812F

[cid:part6.09060806.00000004 at rim.com]




---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20130412/dada30f5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.jpg
Type: image/jpeg
Size: 2318 bytes
Desc: signature.jpg
URL: <http://lists.qt-project.org/pipermail/development/attachments/20130412/dada30f5/attachment.jpg>