[Development] [QtSerialPort] Add some set of base auto tests

Denis Shienkov denis.shienkov at gmail.com
Thu Jan 23 21:03:21 CET 2014


Frederik,

 > The CI team cannot really set up the CI without a single test to try 
running ...

I added some basic tests, please look here: 
https://codereview.qt-project.org/#change,76396

It is enough?

Best regards,
Denis

23.01.2014 21:31, Frederik Gladhorn пишет:
> Torsdag 23. januar 2014 15.52.12 skrev Denis Shienkov:
>>> I think we can start with this, we'll get back when necessary. If you
>> have some test to verify that it works, it would be great.
>>
>> No, unfortunately we have no any tests, because we planned them only after
>> CI will be ready.
> There is really no excuse not to write tests now ;) The CI team cannot really
> set up the CI without a single test to try running, so while they get the
> jenkins bits in place you should write some good tests [1] and [2].
>
> Greetings,
> Frederik
>
> [1] http://qt-project.org/wiki/Writing_Unit_Tests
> [2] http://qt-project.org/wiki/Writing_good_tests
>
>
>> Best regards,
>> Denis
>>
>>
>> 2014/1/23 Fält Simo <Simo.Falt at digia.com>
>>
>>>   On 23.1.2014, at 9.58, Denis Shienkov <denis.shienkov at gmail.com>
>>>   wrote:
>>>   
>>>   Hi Simo, Guys.
>>>
>>> Many thanks for your involvement.
>>>
>>>   > You can use this
>>>
>>> https://bugreports.qt-project.org/browse/QTQAINFRA-682
>>>
>>>   I added there some instruction for "com0com" installation. So, what
>>>
>>> additional info still it is necessary for you?
>>>
>>> I think we can start with this, we'll get back when necessary. If you have
>>> some test to verify that it works, it would be great.
>>>
>>>   Simo
>>>   
>>>   
>>>   Best regards,
>>>
>>> Denis
>>>
>>>
>>> 2014/1/23 Fält Simo <Simo.Falt at digia.com>
>>>
>>>>> -----Original Message-----
>>>>> From: Gladhorn Frederik
>>>>> Sent: 22. tammikuuta 2014 22:19
>>>>> To: Denis Shienkov; development at qt-project.org
>>>>> Cc: Sarajärvi Tony; Fält Simo
>>>>> Subject: RE: [Development] [QtSerialPort] Add some set of base auto
>>>> tests
>>>>
>>>>> On Wednesday 22. January 2014 22.06.03 Denis Shienkov wrote:
>>>>>>   > As mentioned on Gerrit before, I was discussing it with Simo, but
>>>> the
>>>>
>>>>>>   > problem is really that we neither had anything integrated, nor
>>>>>>   > instructions how to set it up... Those are necessary to get done
>>>>>>   > before requesting any further steps on the CI side.
>>>>>>
>>>>>> The short instruction how to setup and configure the "com0com"
>>>> software
>>>>
>>>>>> is here:
>>>>>>
>>>>>> https://codereview.qt-project.org/#change,65431
>>>>>>
>>>>>> For this purpose it is enough to have a host with any Windows x32.
>>>>>> Windows x64 doesn't suit for this purpose because "com0com" is
>>>> delivered
>>>>
>>>>>> with unsigned drivers which won't work there.
>>>>>>
>>>>>> As a result, the principal thing is the configuring of the CI hosts
>>>>>> (setup an ENV variables of QTEST_SERIALPORT_SENDER,
>>>>>> QTEST_SERIALPORT_RECEIVER, and etc.). I don't know how to do it on
>>>>>> CI,
>>>>>> so an help (and some work) from the Digia is necessary, IMHO.
>>>>>>
>>>>>>
>>>>>> Thiago, Oswald, Digia, Others,
>>>>>>
>>>>>> what do you think about it? IMHO, it is impossible to delays with
>>>>>> this
>>>>>> issue more (1~2 years it is too long)...
>>>>>>
>>>>>> If you agree with passes of names of serial ports to tests through
>>>> ENV,
>>>>
>>>>>> then we can start of implementation a set of the minimal tests (even
>>>> if
>>>>
>>>>>> CI is yet ready). Can you comment it?
>>>>> I think writing tests is a great idea in any case, so please go ahead.
>>>>> To get the CI to run the tests we should track this in jira, please
>>>> file an issue
>>>>
>>>>> on the Qt bug tracker under "Qt Quality Assurance Infrastructure".
>>>>   
>>>>   You can use this  https://bugreports.qt-project.org/browse/QTQAINFRA-682
>>>>   
>>>>> As for how to pass the names of the serial ports, it would be best for
>>>> Tony
>>>>
>>>>> and Simo to comment.
>>>>> I would recommend you write a few tests (I see there is currently not
>>>> much
>>>>
>>>>> relevant stuff in tests/auto) as if you had the virtual serial ports
>>>> installed.
>>>>
>>>>   This would help so that we can verify that the setup works.  Configuring
>>>>
>>>> the tests trough ENV variables is quite easy in CI, it is just yet
>>>> another
>>>> parameter in submodule's testconfig. All we need to know are the proper
>>>> values to export.
>>>>
>>>> Simo
>>>>
>>>>> Make sure the tests are skipped for platforms where these are not
>>>> available.
>>>>
>>>>> You can then discuss in the jira task about how to solve the
>>>> installation
>>>>
>>>>> and naming of the serial ports.
>>>>>
>>>>> I'm sure Tony and Simo will set up the needed infrastructure for you
>>>> (with
>>>>
>>>>> your help).
>>>>>
>>>>> Greetings,
>>>>> Frederik
>>>>>
>>>>>> Best regards,
>>>>>> Denis
>>>>>>
>>>>>> 16.01.2014 13:26, Laszlo Papp пишет:
>>>>>>> It had been discussed on this list about 1-2 years ago (if memory
>>>>>>> serves well), and the common consensus was to use com0com on
>>>>> Windows
>>>>>
>>>>>>> and some VT alike option on Linux. I do not think the technology
>>>>>>> changed much, so it is probably still the best option... In my
>>>> opinion
>>>>
>>>>>>> though, the best and easiest option would be socat on Linux. That
>>>> can
>>>>
>>>>>>> elegantly create pseudo terminals for this purpose.
>>>>>>>
>>>>>>> I was writing some unit tests last year targeting the replacement
>>>>>>> of
>>>>>>> socat internally, but it turned out a bit complex, so I lost
>>>>>>> motivation. In theory, it would be possible to use an internal
>>>> "socat"
>>>>
>>>>>>> functionality without requiring external dependencies, so the init
>>>> and
>>>>
>>>>>>> tear down would take over the responsibility for this, but it is
>>>>>>> not
>>>>>>> that urgent. It is probably not an issue on Linux to setup socat,
>>>> but
>>>>
>>>>>>> I will leave that the decision with the CI contributors.
>>>>>>>
>>>>>>> As mentioned on Gerrit before, I was discussing it with Simo, but
>>>> the
>>>>
>>>>>>> problem is really that we neither had anything integrated, nor
>>>>>>> instructions how to set it up... Those are necessary to get done
>>>>>>> before requesting any further steps on the CI side.
>>>>>>>
>>>>>>> In the long future, my opinion is to get QtMock up to the speed,
>>>>>>> preferably using the llvm C++ parser (which was not option back
>>>>>>> then
>>>>>>> when I thought abut it), and then we could remove all this
>>>> workaround
>>>>
>>>>>>> with a cross-platform solution which is not only useful for
>>>>>>> QtSerialPort.
>>>>>>>
>>>>>>> On Tue, Jan 14, 2014 at 8:14 AM, Denis Shienkov
>>>>>>>
>>>>>>> <denis.shienkov at gmail.com> wrote:
>>>>>>>> Hi developers.
>>>>>>>>
>>>>>>>> I want to bring up a question of possibility of addition of some
>>>> base
>>>>
>>>>>>>> tests
>>>>>>>> for QtSerialPort. I understand that it is a complex challenge
>>>> because for
>>>>
>>>>>>>> this purpose is desirable existence of at least two serial ports
>>>>>>>> of
>>>>>>>> devices
>>>>>>>> which are connected by a Null-modem cable.
>>>>>>>>
>>>>>>>> But this problem can be bypassed by use of virtual devices which
>>>> are
>>>>
>>>>>>>> provided with the software of some vendors,
>>>>>>>> e.g.:
>>>>>>>>
>>>>>>>> * on Windows:
>>>>>>>>
>>>>>>>> - eltima virtual serial port driver:
>>>>>>>> http://www.eltima.com/products/vspdxp/
>>>>>>>> - hdd free virtual serial ports:
>>>>>>>> http://www.hhdsoftware.com/free-virtual-serial-ports
>>>>>>>> - com0com null modem emulator:
>>>>> http://sourceforge.net/projects/com0com/
>>>>>
>>>>>>>> and others
>>>>>>>>
>>>>>>>> * on Linux:
>>>>>>>>
>>>>>>>> - tibbo virtual serial port driver:
>>>>>>>> http://tibbo.com/downloads/soi/vspdl.html
>>>>>>>>
>>>>>>>> So, for a covering of the minimum basic tests I would choose
>>>> com0com
>>>>
>>>>> the
>>>>>
>>>>>>>> project (because free) for windows and tibbo the project (but I
>>>>>>>> yet
>>>>>>>> didn't
>>>>>>>> use it never) for linux. In this case on CI it is enough to
>>>> install this
>>>>
>>>>>>>> software and to configure it.
>>>>>>>>
>>>>>>>> The main feature of tests for QtSerialPort is need of explicit
>>>> specifying
>>>>
>>>>>>>> of names of serial ports for their use. So, our team had an idea
>>>> to pass
>>>>
>>>>>>>> these names (two names at least) to tests through ENV.
>>>>>>>>
>>>>>>>> Please see an main ideas example with the tests here:
>>>>>>>> https://codereview.qt-project.org/#change,65431
>>>>>>>>
>>>>>>>> At present behavior of tests such that in case of absence in ENV
>>>> of the
>>>>
>>>>>>>> QTEST_SERIALPORT_SENDER and the QTEST_SERIALPORT_RECEIVER
>>>>> variables any
>>>>>
>>>>>>>> tests will be skipped/ignored. Otherwise each test will take
>>>> necessary
>>>>
>>>>>>>> device name from the necessary variable and to be executed.
>>>>>>>>
>>>>>>>> It allows to use these tests and to developers (for example for
>>>> me) to
>>>>
>>>>>>>> make
>>>>>>>> additional tests that cover the CI tests. Because I can setup as
>>>> the real
>>>>
>>>>>>>> physical serial devices and as other software of the virtual
>>>>>>>> serial
>>>>>>>> devices
>>>>>>>> (for example from Eltima and so forth), i.e. I have an more
>>>> flexibility.
>>>>
>>>>>>>> Also it will be useful for other users who have other types of
>>>> serial
>>>>
>>>>>>>> ports. They can execute these tests with the specific types of
>>>> devices
>>>>
>>>>>>>> that have. Also it will be useful to have these tests available
>>>> even if
>>>>
>>>>>>>> on CI them still isn't present because it will give the chance for
>>>> the
>>>>
>>>>>>>> end users (or for developers) to test changes by means of the same
>>>> test
>>>>
>>>>>>>> sets.
>>>>>>>>
>>>>>>>> Guys, I wait for your comments and sentences about it... Because
>>>> it is
>>>>
>>>>>>>> necessary to solve something, otherwise it is impossible further
>>>>>>>> to
>>>>>>>> continue to work for the project since complexity increases... :(
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Denis
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Development mailing list
>>>>>>>> Development at qt-project.org
>>>>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>> _______________________________________________
>>>>>> Development mailing list
>>>>>> Development at qt-project.org
>>>>>> http://lists.qt-project.org/mailman/listinfo/development




More information about the Development mailing list