[Development] Flakey tests heatmap analysis

[EXT Mitch Curtis]

Hi,

Once I've gotten into a specific filter (i.e. https://testresults.qt.io/grafana/d/de08wetl3yq68f/Test%20failure%20heatmaps?var-integration=149569&var-table_name=blacklisted_tests&var-project=qt%2Fqtdeclarative&var-config=macos-13-arm64-developer-build-tests), is there a way to see a list of test functions (so that I can see which has the most occurrences), or is that not what this is for? I tried clicking on various things but nothing happened.

> -----Original Message-----
> From: Development <development-bounces at qt-project.org> On Behalf Of
> Tor Arne Vestbø via Development
> Sent: Friday, 11 October 2024 10:26 PM
> To: development <development at qt-project.org>
> Subject: [Development] Flakey tests heatmap analysis
> 
> Hi all,
> 
> We had a session during QtCS about flakey tests
> (https://wiki.qt.io/QtCS2024_Flaky) where we discussed how we could gain
> insights about which areas and platforms we are struggling the most.
> 
> Based on Jimis' excellent work on a Postgres database for our test results I
> built a heatmap dashboard that might be helpful in analyzing things:
> 
> 
> 
> https://testresults.qt.io/grafana/d/de08wetl3yq68f/tor-arne-s-heatmap-
> experiment?var-integration=149569&var-table_name=blacklisted_tests&var-
> config=All&var-project=All&orgId=1
> 
> You can click around to drill down, and it also supports listing skipped tests
> (but takes a while to load the data).
> 
> Let me know if there are suggestions on improvements.
> 
> Cheers,
> Tor Arne