[Development] Duplicated test data tags

[Edward Welbourne]

Milian Wolff (Friday, 14 October 2022 3:00 AM)
>> I have many times accidentally written bogus code that duplicated the
>> tags.  Getting a warning is useful, so thanks for working on that!
>>
>> But we won't easily spot these in the thousands of lines of outputs a
>> large test suite is generating.

Indeed, that strikes me as eminently plausible.
Especially as one of the things I've been picking up on (and I've fixed
various of them, at least in QtCore) as a result of chasing these down
is that we have quite a few tests with QWARN messages, that really
should be anticipated (so that the test fails if the warning isn't
produced; and so that the test log isn't cluttered with the warnings)
using QTest::ignoreWarning().

I should also take this opportunity to encourage all developers to watch
out for QWARN messages in test output: if it's unexpected, it may be the
symptom of a lurking bug; otherwise, its presence should be tested for,
see preceding.  If the code under test fails to produce a warning that
was expected, that's an issue the test should catch.

>> At the very least I would suggest something akin to QT_FATAL_WARNINGS
>> that can be set to more easily spot faulty client code.

Perhaps, given the above, we should just encourage developers to use
QT_FATAL_WARNINGS more often ?

I do note, however, that it seems not to have been documented; I shall
add a doc update to my branch.

Mitch Curtis (14 October 2022 03:40) replied:
> QTest::failOnWarning (introduced in 6.3) could also be used by tests
> to make that warning fail the test:
>
> https://doc.qt.io/qt-6/qtest.html#failOnWarning

True enough; but you would need to add that to the start of your _data()
function; I think a global setting is more in line with what's needed.
Fortunately we do have QT_FATAL_WARNINGS; I had forgotten about that.

So the question is: does it suffice to encourage developers to test with
that enabled, or do we need something more specific to the particular
issue of tag and column names in _data() functions ?

	Eddy.