[Development] [QtSerialPort] Add some set of base auto tests

Frederik Gladhorn frederik.gladhorn at digia.com
Fri Jan 24 14:08:35 CET 2014


Fredag 24. januar 2014 00.03.21 skrev Denis Shienkov:
> Frederik,
> 
>  > The CI team cannot really set up the CI without a single test to try
> 
> running ...
> 
> I added some basic tests, please look here:
> https://codereview.qt-project.org/#change,76396

Great, I think this makes setting up the CI much easier since there is a first 
verification that things are working.

Thanks for the quick response :)

Greetings,
Frederik

> 
> It is enough?
> 
> Best regards,
> Denis
> 
> 23.01.2014 21:31, Frederik Gladhorn пишет:
> > Torsdag 23. januar 2014 15.52.12 skrev Denis Shienkov:
> >>> I think we can start with this, we'll get back when necessary. If you
> >> 
> >> have some test to verify that it works, it would be great.
> >> 
> >> No, unfortunately we have no any tests, because we planned them only
> >> after
> >> CI will be ready.
> > 
> > There is really no excuse not to write tests now ;) The CI team cannot
> > really set up the CI without a single test to try running, so while they
> > get the jenkins bits in place you should write some good tests [1] and
> > [2].
> > 
> > Greetings,
> > Frederik
> > 
> > [1] http://qt-project.org/wiki/Writing_Unit_Tests
> > [2] http://qt-project.org/wiki/Writing_good_tests
> > 
> >> Best regards,
> >> Denis
> >> 
> >> 
> >> 2014/1/23 Fält Simo <Simo.Falt at digia.com>
> >> 
> >>>   On 23.1.2014, at 9.58, Denis Shienkov <denis.shienkov at gmail.com>
> >>>   wrote:
> >>>   
> >>>   Hi Simo, Guys.
> >>> 
> >>> Many thanks for your involvement.
> >>> 
> >>>   > You can use this
> >>> 
> >>> https://bugreports.qt-project.org/browse/QTQAINFRA-682
> >>> 
> >>>   I added there some instruction for "com0com" installation. So, what
> >>> 
> >>> additional info still it is necessary for you?
> >>> 
> >>> I think we can start with this, we'll get back when necessary. If you
> >>> have
> >>> some test to verify that it works, it would be great.
> >>> 
> >>>   Simo
> >>>   
> >>>   
> >>>   Best regards,
> >>> 
> >>> Denis
> >>> 
> >>> 
> >>> 2014/1/23 Fält Simo <Simo.Falt at digia.com>
> >>> 
> >>>>> -----Original Message-----
> >>>>> From: Gladhorn Frederik
> >>>>> Sent: 22. tammikuuta 2014 22:19
> >>>>> To: Denis Shienkov; development at qt-project.org
> >>>>> Cc: Sarajärvi Tony; Fält Simo
> >>>>> Subject: RE: [Development] [QtSerialPort] Add some set of base auto
> >>>> 
> >>>> tests
> >>>> 
> >>>>> On Wednesday 22. January 2014 22.06.03 Denis Shienkov wrote:
> >>>>>>   > As mentioned on Gerrit before, I was discussing it with Simo, but
> >>>> 
> >>>> the
> >>>> 
> >>>>>>   > problem is really that we neither had anything integrated, nor
> >>>>>>   > instructions how to set it up... Those are necessary to get done
> >>>>>>   > before requesting any further steps on the CI side.
> >>>>>> 
> >>>>>> The short instruction how to setup and configure the "com0com"
> >>>> 
> >>>> software
> >>>> 
> >>>>>> is here:
> >>>>>> 
> >>>>>> https://codereview.qt-project.org/#change,65431
> >>>>>> 
> >>>>>> For this purpose it is enough to have a host with any Windows x32.
> >>>>>> Windows x64 doesn't suit for this purpose because "com0com" is
> >>>> 
> >>>> delivered
> >>>> 
> >>>>>> with unsigned drivers which won't work there.
> >>>>>> 
> >>>>>> As a result, the principal thing is the configuring of the CI hosts
> >>>>>> (setup an ENV variables of QTEST_SERIALPORT_SENDER,
> >>>>>> QTEST_SERIALPORT_RECEIVER, and etc.). I don't know how to do it on
> >>>>>> CI,
> >>>>>> so an help (and some work) from the Digia is necessary, IMHO.
> >>>>>> 
> >>>>>> 
> >>>>>> Thiago, Oswald, Digia, Others,
> >>>>>> 
> >>>>>> what do you think about it? IMHO, it is impossible to delays with
> >>>>>> this
> >>>>>> issue more (1~2 years it is too long)...
> >>>>>> 
> >>>>>> If you agree with passes of names of serial ports to tests through
> >>>> 
> >>>> ENV,
> >>>> 
> >>>>>> then we can start of implementation a set of the minimal tests (even
> >>>> 
> >>>> if
> >>>> 
> >>>>>> CI is yet ready). Can you comment it?
> >>>>> 
> >>>>> I think writing tests is a great idea in any case, so please go ahead.
> >>>>> To get the CI to run the tests we should track this in jira, please
> >>>> 
> >>>> file an issue
> >>>> 
> >>>>> on the Qt bug tracker under "Qt Quality Assurance Infrastructure".
> >>>>> 
> >>>>   You can use this 
> >>>>   https://bugreports.qt-project.org/browse/QTQAINFRA-682
> >>>>> 
> >>>>> As for how to pass the names of the serial ports, it would be best for
> >>>> 
> >>>> Tony
> >>>> 
> >>>>> and Simo to comment.
> >>>>> I would recommend you write a few tests (I see there is currently not
> >>>> 
> >>>> much
> >>>> 
> >>>>> relevant stuff in tests/auto) as if you had the virtual serial ports
> >>>> 
> >>>> installed.
> >>>> 
> >>>>   This would help so that we can verify that the setup works. 
> >>>>   Configuring
> >>>> 
> >>>> the tests trough ENV variables is quite easy in CI, it is just yet
> >>>> another
> >>>> parameter in submodule's testconfig. All we need to know are the proper
> >>>> values to export.
> >>>> 
> >>>> Simo
> >>>> 
> >>>>> Make sure the tests are skipped for platforms where these are not
> >>>> 
> >>>> available.
> >>>> 
> >>>>> You can then discuss in the jira task about how to solve the
> >>>> 
> >>>> installation
> >>>> 
> >>>>> and naming of the serial ports.
> >>>>> 
> >>>>> I'm sure Tony and Simo will set up the needed infrastructure for you
> >>>> 
> >>>> (with
> >>>> 
> >>>>> your help).
> >>>>> 
> >>>>> Greetings,
> >>>>> Frederik
> >>>>> 
> >>>>>> Best regards,
> >>>>>> Denis
> >>>>>> 
> >>>>>> 16.01.2014 13:26, Laszlo Papp пишет:
> >>>>>>> It had been discussed on this list about 1-2 years ago (if memory
> >>>>>>> serves well), and the common consensus was to use com0com on
> >>>>> 
> >>>>> Windows
> >>>>> 
> >>>>>>> and some VT alike option on Linux. I do not think the technology
> >>>>>>> changed much, so it is probably still the best option... In my
> >>>> 
> >>>> opinion
> >>>> 
> >>>>>>> though, the best and easiest option would be socat on Linux. That
> >>>> 
> >>>> can
> >>>> 
> >>>>>>> elegantly create pseudo terminals for this purpose.
> >>>>>>> 
> >>>>>>> I was writing some unit tests last year targeting the replacement
> >>>>>>> of
> >>>>>>> socat internally, but it turned out a bit complex, so I lost
> >>>>>>> motivation. In theory, it would be possible to use an internal
> >>>> 
> >>>> "socat"
> >>>> 
> >>>>>>> functionality without requiring external dependencies, so the init
> >>>> 
> >>>> and
> >>>> 
> >>>>>>> tear down would take over the responsibility for this, but it is
> >>>>>>> not
> >>>>>>> that urgent. It is probably not an issue on Linux to setup socat,
> >>>> 
> >>>> but
> >>>> 
> >>>>>>> I will leave that the decision with the CI contributors.
> >>>>>>> 
> >>>>>>> As mentioned on Gerrit before, I was discussing it with Simo, but
> >>>> 
> >>>> the
> >>>> 
> >>>>>>> problem is really that we neither had anything integrated, nor
> >>>>>>> instructions how to set it up... Those are necessary to get done
> >>>>>>> before requesting any further steps on the CI side.
> >>>>>>> 
> >>>>>>> In the long future, my opinion is to get QtMock up to the speed,
> >>>>>>> preferably using the llvm C++ parser (which was not option back
> >>>>>>> then
> >>>>>>> when I thought abut it), and then we could remove all this
> >>>> 
> >>>> workaround
> >>>> 
> >>>>>>> with a cross-platform solution which is not only useful for
> >>>>>>> QtSerialPort.
> >>>>>>> 
> >>>>>>> On Tue, Jan 14, 2014 at 8:14 AM, Denis Shienkov
> >>>>>>> 
> >>>>>>> <denis.shienkov at gmail.com> wrote:
> >>>>>>>> Hi developers.
> >>>>>>>> 
> >>>>>>>> I want to bring up a question of possibility of addition of some
> >>>> 
> >>>> base
> >>>> 
> >>>>>>>> tests
> >>>>>>>> for QtSerialPort. I understand that it is a complex challenge
> >>>> 
> >>>> because for
> >>>> 
> >>>>>>>> this purpose is desirable existence of at least two serial ports
> >>>>>>>> of
> >>>>>>>> devices
> >>>>>>>> which are connected by a Null-modem cable.
> >>>>>>>> 
> >>>>>>>> But this problem can be bypassed by use of virtual devices which
> >>>> 
> >>>> are
> >>>> 
> >>>>>>>> provided with the software of some vendors,
> >>>>>>>> e.g.:
> >>>>>>>> 
> >>>>>>>> * on Windows:
> >>>>>>>> 
> >>>>>>>> - eltima virtual serial port driver:
> >>>>>>>> http://www.eltima.com/products/vspdxp/
> >>>>>>>> - hdd free virtual serial ports:
> >>>>>>>> http://www.hhdsoftware.com/free-virtual-serial-ports
> >>>>> 
> >>>>>>>> - com0com null modem emulator:
> >>>>> http://sourceforge.net/projects/com0com/
> >>>>> 
> >>>>>>>> and others
> >>>>>>>> 
> >>>>>>>> * on Linux:
> >>>>>>>> 
> >>>>>>>> - tibbo virtual serial port driver:
> >>>>>>>> http://tibbo.com/downloads/soi/vspdl.html
> >>>>>>>> 
> >>>>>>>> So, for a covering of the minimum basic tests I would choose
> >>>> 
> >>>> com0com
> >>>> 
> >>>>> the
> >>>>> 
> >>>>>>>> project (because free) for windows and tibbo the project (but I
> >>>>>>>> yet
> >>>>>>>> didn't
> >>>>>>>> use it never) for linux. In this case on CI it is enough to
> >>>> 
> >>>> install this
> >>>> 
> >>>>>>>> software and to configure it.
> >>>>>>>> 
> >>>>>>>> The main feature of tests for QtSerialPort is need of explicit
> >>>> 
> >>>> specifying
> >>>> 
> >>>>>>>> of names of serial ports for their use. So, our team had an idea
> >>>> 
> >>>> to pass
> >>>> 
> >>>>>>>> these names (two names at least) to tests through ENV.
> >>>>>>>> 
> >>>>>>>> Please see an main ideas example with the tests here:
> >>>>>>>> https://codereview.qt-project.org/#change,65431
> >>>>>>>> 
> >>>>>>>> At present behavior of tests such that in case of absence in ENV
> >>>> 
> >>>> of the
> >>>> 
> >>>>>>>> QTEST_SERIALPORT_SENDER and the QTEST_SERIALPORT_RECEIVER
> >>>>> 
> >>>>> variables any
> >>>>> 
> >>>>>>>> tests will be skipped/ignored. Otherwise each test will take
> >>>> 
> >>>> necessary
> >>>> 
> >>>>>>>> device name from the necessary variable and to be executed.
> >>>>>>>> 
> >>>>>>>> It allows to use these tests and to developers (for example for
> >>>> 
> >>>> me) to
> >>>> 
> >>>>>>>> make
> >>>>>>>> additional tests that cover the CI tests. Because I can setup as
> >>>> 
> >>>> the real
> >>>> 
> >>>>>>>> physical serial devices and as other software of the virtual
> >>>>>>>> serial
> >>>>>>>> devices
> >>>>>>>> (for example from Eltima and so forth), i.e. I have an more
> >>>> 
> >>>> flexibility.
> >>>> 
> >>>>>>>> Also it will be useful for other users who have other types of
> >>>> 
> >>>> serial
> >>>> 
> >>>>>>>> ports. They can execute these tests with the specific types of
> >>>> 
> >>>> devices
> >>>> 
> >>>>>>>> that have. Also it will be useful to have these tests available
> >>>> 
> >>>> even if
> >>>> 
> >>>>>>>> on CI them still isn't present because it will give the chance for
> >>>> 
> >>>> the
> >>>> 
> >>>>>>>> end users (or for developers) to test changes by means of the same
> >>>> 
> >>>> test
> >>>> 
> >>>>>>>> sets.
> >>>>>>>> 
> >>>>>>>> Guys, I wait for your comments and sentences about it... Because
> >>>> 
> >>>> it is
> >>>> 
> >>>>>>>> necessary to solve something, otherwise it is impossible further
> >>>>>>>> to
> >>>>>>>> continue to work for the project since complexity increases... :(
> >>>>>>>> 
> >>>>>>>> Best regards,
> >>>>>>>> Denis
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> _______________________________________________
> >>>>>>>> Development mailing list
> >>>>>>>> Development at qt-project.org
> >>>>>>>> http://lists.qt-project.org/mailman/listinfo/development
> >>>>>> 
> >>>>>> _______________________________________________
> >>>>>> Development mailing list
> >>>>>> Development at qt-project.org
> >>>>>> http://lists.qt-project.org/mailman/listinfo/development

-- 
Best regards,
Frederik Gladhorn
Senior Software Engineer - Digia, Qt
Visit us on: http://qt.digia.com




More information about the Development mailing list