[Development] Introducing QtMetrics

[Sarajärvi Tony]

Hi

If I recall I already limited the amount of logs to be scanned so that the scanner only scanned logs dated after Feb 1st 2013. Also every entry in the database is date-stamped, so I can remove old entries from time to time. However, I don't personally know which branches are obsolete, so I'm only playing with dates.

If we look at all the logs in testresults.qt-project.org/ci, those are periodically cleaned by someone of us. Cleaned means that all old logs since some date are archived.

You also have the possibility to filter accordingly to time, but I'm unsure if that fulfills your needs in this case.


Then a bit off topic. The Qt Metrics page lists the build status and test status accordingly to the state of the test set (tst_xxxxx). This however might not be precise enough. If we go and make the whole test set insignificant, we remove several good cases at the same time.

I already improved the log scanner, and I have another database already existing, where I store this data per test case, not test set. No GUI is available for that however, and could take a while until someone has time to change or add this on top of Qt Metrics. 

I manually gathered this piece of information based on this new information: http://qt-project.org/wiki/CI_Failing_Tests
The page was not mine, but I added that new latest information there. It also has interesting information regarding Flaky cases. With that amount of flaky cases that are re-run to get them passing, I'm wondering how much time would be saved per build, if these tests wouldn't have to be  re-run to get them passing ;)

-Tony

> -----Original Message-----
> From: Stephen Kelly [mailto:stephen.kelly at kdab.com]
> Sent: 5. heinäkuuta 2013 13:49
> To: development at qt-project.org; Sarajärvi Tony
> Subject: Re: [Development] Introducing QtMetrics
> 
> On Tuesday, June 04, 2013 10:13:16 Sarajärvi Tony wrote:
> > Hi
> >
> > We are ready to launch our QtMetrics page for the public now.
> >
> > The main page can be found here:
> > http://testresults.qt-project.org/qtmetrics/index.php
> 
> Thanks for this!
> 
> I've tried it out and I noticed that there is a lot of noise from obsolete
> configurations. For example, after the page loaded, I selected the cmake test
> from the third drop-down. That lists some significant failures, however, all
> failures are in obsolete master branches.
> 
> The color-coded table below also contains similiar noisy lines of obsolete
> master branches, and of repos which are not part of Qt 5 (eg QtJsonDb).
> 
> Can all this stuff be filtered out or removed somehow?
> 
> Thanks,
> 
> --
> Join us in October at Qt Developer Days 2013 - https://devdays.kdab.com
> 
> Stephen Kelly <stephen.kelly at kdab.com> | Software Engineer
> KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
> www.kdab.com || Germany +49-30-521325470 || Sweden (HQ) +46-563-
> 540090
> KDAB - Qt Experts - Platform-Independent Software Solutions