[Development] [QtSerialPort] Add some set of base auto tests

Frederik Gladhorn frederik.gladhorn at digia.com
Thu Jan 23 18:31:41 CET 2014 

Torsdag 23. januar 2014 15.52.12 skrev Denis Shienkov:
> > I think we can start with this, we'll get back when necessary. If you
> 
> have some test to verify that it works, it would be great.
> 
> No, unfortunately we have no any tests, because we planned them only after
> CI will be ready.

There is really no excuse not to write tests now ;) The CI team cannot really 
set up the CI without a single test to try running, so while they get the 
jenkins bits in place you should write some good tests [1] and [2].

Greetings,
Frederik

[1] http://qt-project.org/wiki/Writing_Unit_Tests
[2] http://qt-project.org/wiki/Writing_good_tests


> 
> Best regards,
> Denis
> 
> 
> 2014/1/23 Fält Simo <Simo.Falt at digia.com>
> 
> >  On 23.1.2014, at 9.58, Denis Shienkov <denis.shienkov at gmail.com>
> >  wrote:
> >  
> >  Hi Simo, Guys.
> > 
> > Many thanks for your involvement.
> > 
> >  > You can use this
> > 
> > https://bugreports.qt-project.org/browse/QTQAINFRA-682
> > 
> >  I added there some instruction for "com0com" installation. So, what
> > 
> > additional info still it is necessary for you?
> > 
> > I think we can start with this, we'll get back when necessary. If you have
> > some test to verify that it works, it would be great.
> > 
> >  Simo
> >  
> >  
> >  Best regards,
> > 
> > Denis
> > 
> > 
> > 2014/1/23 Fält Simo <Simo.Falt at digia.com>
> > 
> >> > -----Original Message-----
> >> > From: Gladhorn Frederik
> >> > Sent: 22. tammikuuta 2014 22:19
> >> > To: Denis Shienkov; development at qt-project.org
> >> > Cc: Sarajärvi Tony; Fält Simo
> >> > Subject: RE: [Development] [QtSerialPort] Add some set of base auto
> >> 
> >> tests
> >> 
> >> > On Wednesday 22. January 2014 22.06.03 Denis Shienkov wrote:
> >> > >  > As mentioned on Gerrit before, I was discussing it with Simo, but
> >> 
> >> the
> >> 
> >> > >  > problem is really that we neither had anything integrated, nor
> >> > >  > instructions how to set it up... Those are necessary to get done
> >> > >  > before requesting any further steps on the CI side.
> >> > > 
> >> > > The short instruction how to setup and configure the "com0com"
> >> 
> >> software
> >> 
> >> > > is here:
> >> > > 
> >> > > https://codereview.qt-project.org/#change,65431
> >> > > 
> >> > > For this purpose it is enough to have a host with any Windows x32.
> >> > > Windows x64 doesn't suit for this purpose because "com0com" is
> >> 
> >> delivered
> >> 
> >> > > with unsigned drivers which won't work there.
> >> > > 
> >> > > As a result, the principal thing is the configuring of the CI hosts
> >> > > (setup an ENV variables of QTEST_SERIALPORT_SENDER,
> >> > > QTEST_SERIALPORT_RECEIVER, and etc.). I don't know how to do it on
> >> > > CI,
> >> > > so an help (and some work) from the Digia is necessary, IMHO.
> >> > > 
> >> > > 
> >> > > Thiago, Oswald, Digia, Others,
> >> > > 
> >> > > what do you think about it? IMHO, it is impossible to delays with
> >> > > this
> >> > > issue more (1~2 years it is too long)...
> >> > > 
> >> > > If you agree with passes of names of serial ports to tests through
> >> 
> >> ENV,
> >> 
> >> > > then we can start of implementation a set of the minimal tests (even
> >> 
> >> if
> >> 
> >> > > CI is yet ready). Can you comment it?
> >> > 
> >> > I think writing tests is a great idea in any case, so please go ahead.
> >> > To get the CI to run the tests we should track this in jira, please
> >> 
> >> file an issue
> >> 
> >> > on the Qt bug tracker under "Qt Quality Assurance Infrastructure".
> >>  
> >>  You can use this  https://bugreports.qt-project.org/browse/QTQAINFRA-682
> >>  
> >> > As for how to pass the names of the serial ports, it would be best for
> >> 
> >> Tony
> >> 
> >> > and Simo to comment.
> >> > I would recommend you write a few tests (I see there is currently not
> >> 
> >> much
> >> 
> >> > relevant stuff in tests/auto) as if you had the virtual serial ports
> >> 
> >> installed.
> >> 
> >>  This would help so that we can verify that the setup works.  Configuring
> >> 
> >> the tests trough ENV variables is quite easy in CI, it is just yet
> >> another
> >> parameter in submodule's testconfig. All we need to know are the proper
> >> values to export.
> >> 
> >> Simo
> >> 
> >> > Make sure the tests are skipped for platforms where these are not
> >> 
> >> available.
> >> 
> >> > You can then discuss in the jira task about how to solve the
> >> 
> >> installation
> >> 
> >> > and naming of the serial ports.
> >> > 
> >> > I'm sure Tony and Simo will set up the needed infrastructure for you
> >> 
> >> (with
> >> 
> >> > your help).
> >> > 
> >> > Greetings,
> >> > Frederik
> >> > 
> >> > > Best regards,
> >> > > Denis
> >> > > 
> >> > > 16.01.2014 13:26, Laszlo Papp пишет:
> >> > > > It had been discussed on this list about 1-2 years ago (if memory
> >> > > > serves well), and the common consensus was to use com0com on
> >> > 
> >> > Windows
> >> > 
> >> > > > and some VT alike option on Linux. I do not think the technology
> >> > > > changed much, so it is probably still the best option... In my
> >> 
> >> opinion
> >> 
> >> > > > though, the best and easiest option would be socat on Linux. That
> >> 
> >> can
> >> 
> >> > > > elegantly create pseudo terminals for this purpose.
> >> > > > 
> >> > > > I was writing some unit tests last year targeting the replacement
> >> > > > of
> >> > > > socat internally, but it turned out a bit complex, so I lost
> >> > > > motivation. In theory, it would be possible to use an internal
> >> 
> >> "socat"
> >> 
> >> > > > functionality without requiring external dependencies, so the init
> >> 
> >> and
> >> 
> >> > > > tear down would take over the responsibility for this, but it is
> >> > > > not
> >> > > > that urgent. It is probably not an issue on Linux to setup socat,
> >> 
> >> but
> >> 
> >> > > > I will leave that the decision with the CI contributors.
> >> > > > 
> >> > > > As mentioned on Gerrit before, I was discussing it with Simo, but
> >> 
> >> the
> >> 
> >> > > > problem is really that we neither had anything integrated, nor
> >> > > > instructions how to set it up... Those are necessary to get done
> >> > > > before requesting any further steps on the CI side.
> >> > > > 
> >> > > > In the long future, my opinion is to get QtMock up to the speed,
> >> > > > preferably using the llvm C++ parser (which was not option back
> >> > > > then
> >> > > > when I thought abut it), and then we could remove all this
> >> 
> >> workaround
> >> 
> >> > > > with a cross-platform solution which is not only useful for
> >> > > > QtSerialPort.
> >> > > > 
> >> > > > On Tue, Jan 14, 2014 at 8:14 AM, Denis Shienkov
> >> > > > 
> >> > > > <denis.shienkov at gmail.com> wrote:
> >> > > >> Hi developers.
> >> > > >> 
> >> > > >> I want to bring up a question of possibility of addition of some
> >> 
> >> base
> >> 
> >> > > >> tests
> >> > > >> for QtSerialPort. I understand that it is a complex challenge
> >> 
> >> because for
> >> 
> >> > > >> this purpose is desirable existence of at least two serial ports
> >> > > >> of
> >> > > >> devices
> >> > > >> which are connected by a Null-modem cable.
> >> > > >> 
> >> > > >> But this problem can be bypassed by use of virtual devices which
> >> 
> >> are
> >> 
> >> > > >> provided with the software of some vendors,
> >> > > >> e.g.:
> >> > > >> 
> >> > > >> * on Windows:
> >> > > >> 
> >> > > >> - eltima virtual serial port driver:
> >> > > >> http://www.eltima.com/products/vspdxp/
> >> > > >> - hdd free virtual serial ports:
> >> > > >> http://www.hhdsoftware.com/free-virtual-serial-ports
> >> > 
> >> > > >> - com0com null modem emulator:
> >> > http://sourceforge.net/projects/com0com/
> >> > 
> >> > > >> and others
> >> > > >> 
> >> > > >> * on Linux:
> >> > > >> 
> >> > > >> - tibbo virtual serial port driver:
> >> > > >> http://tibbo.com/downloads/soi/vspdl.html
> >> > > >> 
> >> > > >> So, for a covering of the minimum basic tests I would choose
> >> 
> >> com0com
> >> 
> >> > the
> >> > 
> >> > > >> project (because free) for windows and tibbo the project (but I
> >> > > >> yet
> >> > > >> didn't
> >> > > >> use it never) for linux. In this case on CI it is enough to
> >> 
> >> install this
> >> 
> >> > > >> software and to configure it.
> >> > > >> 
> >> > > >> The main feature of tests for QtSerialPort is need of explicit
> >> 
> >> specifying
> >> 
> >> > > >> of names of serial ports for their use. So, our team had an idea
> >> 
> >> to pass
> >> 
> >> > > >> these names (two names at least) to tests through ENV.
> >> > > >> 
> >> > > >> Please see an main ideas example with the tests here:
> >> > > >> https://codereview.qt-project.org/#change,65431
> >> > > >> 
> >> > > >> At present behavior of tests such that in case of absence in ENV
> >> 
> >> of the
> >> 
> >> > > >> QTEST_SERIALPORT_SENDER and the QTEST_SERIALPORT_RECEIVER
> >> > 
> >> > variables any
> >> > 
> >> > > >> tests will be skipped/ignored. Otherwise each test will take
> >> 
> >> necessary
> >> 
> >> > > >> device name from the necessary variable and to be executed.
> >> > > >> 
> >> > > >> It allows to use these tests and to developers (for example for
> >> 
> >> me) to
> >> 
> >> > > >> make
> >> > > >> additional tests that cover the CI tests. Because I can setup as
> >> 
> >> the real
> >> 
> >> > > >> physical serial devices and as other software of the virtual
> >> > > >> serial
> >> > > >> devices
> >> > > >> (for example from Eltima and so forth), i.e. I have an more
> >> 
> >> flexibility.
> >> 
> >> > > >> Also it will be useful for other users who have other types of
> >> 
> >> serial
> >> 
> >> > > >> ports. They can execute these tests with the specific types of
> >> 
> >> devices
> >> 
> >> > > >> that have. Also it will be useful to have these tests available
> >> 
> >> even if
> >> 
> >> > > >> on CI them still isn't present because it will give the chance for
> >> 
> >> the
> >> 
> >> > > >> end users (or for developers) to test changes by means of the same
> >> 
> >> test
> >> 
> >> > > >> sets.
> >> > > >> 
> >> > > >> Guys, I wait for your comments and sentences about it... Because
> >> 
> >> it is
> >> 
> >> > > >> necessary to solve something, otherwise it is impossible further
> >> > > >> to
> >> > > >> continue to work for the project since complexity increases... :(
> >> > > >> 
> >> > > >> Best regards,
> >> > > >> Denis
> >> > > >> 
> >> > > >> 
> >> > > >> 
> >> > > >> 
> >> > > >> _______________________________________________
> >> > > >> Development mailing list
> >> > > >> Development at qt-project.org
> >> > > >> http://lists.qt-project.org/mailman/listinfo/development
> >> > > 
> >> > > _______________________________________________
> >> > > Development mailing list
> >> > > Development at qt-project.org
> >> > > http://lists.qt-project.org/mailman/listinfo/development

-- 
Best regards,
Frederik Gladhorn
Senior Software Engineer - Digia, Qt
Visit us on: http://qt.digia.com

More information about the Development mailing list