[Development] Supporting helper functions in auto tests by providing throwing Qt Test macros

[Giuseppe D'Angelo]

Il 03/04/19 13:58, Mitch Curtis ha scritto:
>>> I have a question regarding this particular implementation, that is,
>>> why adding dedicated testing functions? Can't you "simply" change the
>>> currently existing QVERIFY/QCOMPARE to throw exceptions in case of
>>> failure, instead of returning from the current test function? This
>>> would achieve major consistency -- simply use the same functions
>> everywhere, no matter what.
>>
>> To me, a change like this (making the existing macros throw) seems more like
>> a Qt 6 thing, but I could be wrong. I only went with new macros because I
>> was afraid of breaking stuff, but yeah, just changing the existing ones would
>> be ideal.

If you expect Qt 6 to be as much source compatible with Qt 5 as 
possible, then you can make this change right now as much you can make 
it in Qt 6.

So, let's discuss: what breaks if you make today's macros throw instead 
of return?


>>
>>> This idea isn't failproof, as:
>>>
>>> 1) it requires the exception to propagate through a direct
>>> QMetaObject::invoke (that is, whatever mechanism testlib uses to
>>> invoke the test functions); but that should be already supported and
>> tested, even.
>>
>> Hmm, I hadn't thought about invoke()... hopefully that's not an (unsolvable)
>> issue, otherwise it kinda ruins this whole idea.

As I said, I believe these codepaths support exceptions (but it's not 
exactly documented). If they don't we can should fix them to do so.


>>> 2) it causes a minor SC in case the user was doing it wrong --
>>> "catching all", without rethrowing an unknown exception type. Before,
>>> a failing test would simply return. With this idea, it would throw,
>>> and the exception could be caught by the catch all and not used to
>>> block the execution of the current test function.
>>>
>>> The last can be solved in a very simple way (e.g. make this mechanism
>>> opt-in; and also provide something to the user to test if the current
>>> catch-all is a Qt test failure, via std::current_exception()).
>>
>> What kind of opt-in mechanism did you have in mind?

Anything that is suitable -- env variables, a different macro to create 
the test's main, something to set in your initTestCase(), a CONFIG 
switch in your .pro, etc.

Cheers,
-- 
Giuseppe D'Angelo | giuseppe.dangelo at kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4007 bytes
Desc: Firma crittografica S/MIME
URL: <http://lists.qt-project.org/pipermail/development/attachments/20190405/e307da32/attachment.bin>