[Development] qmllint's MaxWarnings and how to detect commits introducing new qmllint warnings

[Sami Shalayel]

Hi everyone,


TL;DR: How do you detect commits that introduce new qmllint warnings in 
your projects, and do you need a mechanism for that in qmllint?

To detect newly introduced qmllint warning in user QML code, qmllint 
reads MaxWarnings values from .qmllint.ini files in the source folders. 
The feature was made I believe for situations where QML code is not 
perfect yet and has already an already existing number of issues/QML 
warnings, but where introducing new qmllint warnings should fail the build.

One of the shortcomings of this feature is that qmllint's build target 
runs qmllint on multiple QML files, and uses the same MaxWarnings for 
all QML files. For example, a project with 3 files A.qml, B.qml, and 
C.qml is "regression-clean" with MaxWarnings=5 if all QML files have 
less than 5 warnings.

This is not really helpful in some cases, for example qmllint will 
currently not flag newly introduced qmllint warnings on a 
CleanNewCode.qml file that used to have 0 qmllint warnings if this QML 
file shares the same MaxWarnings as a AwfulLegacyDontTouch.qml file 
which needs MaxWarnings=999 because both QML files are in the same QML 
Module.

To remedy this, we could add a new .qmllint.ini entry called 
TotalWarningCount (or use a separate ini file), which makes sure that 
the sum of all qmllint warnings of all linted files stay under a total 
warning. This has the benefit of flagging newly introduced regressions 
on files with less warnings than MaxWarnings, but I see two downsides:

 1.

    We don't see "replacements", for example a commit removing a warning
    but introducing another is not flagged by this strategy (and is also
    not flagged with the already-existing implementation).

 2.

    .qmllint.ini can be anywhere in the source folder. For example, if
    you have 5 QML modules sharing the same .qmllint.ini, then all 5 QML
    modules will have to share the same TotalWarningCount and
    MaxWarnings, even if only one QML module has legacy QML code with
    lots of warnings. And duplicating .qmllint.ini might be tedious when
    having a custom warning setup, so maybe this will require
    introducing TotalWarningCount via a new separate .ini file?

An alternative would be to implement some baseline support to qmllint, 
similar to 
https://www.jetbrains.com/help/qodana/docker-image-configuration.html#docker-config-reference-basel… 
<https://www.jetbrains.com/help/qodana/docker-image-configuration.html#docker-config-reference-baseline> 
or https://docs.basedpyright.com/latest/benefits-over-pyright/baseline/ 
, where qmllint warnings would be stored somewhere (probably in the 
source folder), and |qmllint --fail-on-regressions ...| and |qmllint 
--update-baseline|could could be used to fail when new warnings are 
introduced, and to create or update the baseline.

Advantages of this alternative would be that all 
regressions/improvements could be caught, including false positions when 
moving code around for example. In addition, we could also extend 
qmllint with some |qmllint --only-regressions| flag to only show new 
warnings, so that legacy code warnings are not shown when working on new 
code.

Disadvantages would be that some CMake target 
|qt_qmllint_update_baseline| might be required to write the baseline 
into the source folder. Maybe this "source-folder-tampering" is 
acceptable if this target is manual only and never triggered 
automatically by another target?

How do you track and find commits that introduce new qmllint warnings in 
your projects, and do you need a mechanism for that in qmllint? If yes, 
which parts of TotalWarningCount or the baseline approaches might be 
useful for your project or use-case?

Best regards,

Sami Shalayel


PS: You can also find the suggestion on JIRA: 
https://qt-project.atlassian.net/browse/QTBUG-148801.

-- 



Sami Shalayel

he/him

Specialist Software Engineer



The Qt Company GmbH
Erich-Thilo-Str. 10
12489 Berlin, Germany

sami.shalayel at qt.io

+49 176 15967267

www.qt.io <https://www.qt.io>



Geschäftsführer: Mika Pälsi, Juha Varelius, Jouni Lintunen Sitz der 
Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 
144331 B




https://www.qt.io/ <https://www.qt.io/>
https://www.facebook.com/qt/ <https://www.facebook.com/qt/> 
https://twitter.com/qtproject <https://twitter.com/qtproject> 
https://www.linkedin.com/company/qtgroup/ 
<https://www.linkedin.com/company/qtgroup/> 
https://www.youtube.com/QtStudios <https://www.youtube.com/QtStudios>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20260804/aff5158e/attachment.htm>