[Development] Flakey tests heatmap analysis

[Tor Arne Vestbø]

Hi all,

We had a session during QtCS about flakey tests (https://wiki.qt.io/QtCS2024_Flaky) where we discussed how we could gain insights about which areas and platforms we are struggling the most.

Based on Jimis’ excellent work on a Postgres database for our test results I built a heatmap dashboard that might be helpful in analyzing things:

[PastedGraphic-2.png]

https://testresults.qt.io/grafana/d/de08wetl3yq68f/tor-arne-s-heatmap-experiment?var-integration=149569&var-table_name=blacklisted_tests&var-config=All&var-project=All&orgId=1

You can click around to drill down, and it also supports listing skipped tests (but takes a while to load the data).

Let me know if there are suggestions on improvements.

Cheers,
Tor Arne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20241011/2fc068ff/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-2.png
Type: image/png
Size: 916240 bytes
Desc: PastedGraphic-2.png
URL: <http://lists.qt-project.org/pipermail/development/attachments/20241011/2fc068ff/attachment-0001.png>