[Qt-creator] Editors/Code-models: new year's resolutions

[Stenger Christian]

Hi Przemek,

regarding Squish you'll need to have a valid Squish license to run/create Squish tests.
All squish tests we're using inside CI are actually written in Python and can be found inside the Creator repository. (see $REPO/tests/system)

There are some pre-requisites to be able to (fully) run these tests - most of them should be listed inside the README file within the same folder.

Best regards,
Christian


________________________________
From: Przemyslaw Gorszkowski [pgorszkowski at gmail.com]
Sent: Monday, January 14, 2013 12:05 PM
To: Kosjar Nikolai
Cc: qt-creator at qt-project.org; Stenger Christian; Loehning Robert
Subject: Re: [Qt-creator] Editors/Code-models: new year's resolutions

Hi Nikolai,

Thank you very much for this information. Esspecially about "External auto tests". I didn't know about them.
Your reply covers almost all my questions regarding to point 1.
It would be good to get some more info about Squish and possibilities here for "external" developers.

Best Regards,

Przemek

On Mon, Jan 14, 2013 at 10:19 AM, Nikolai Kosjar <nikolai.kosjar at digia.com<mailto:nikolai.kosjar at digia.com>> wrote:
Hi!

On 01/12/2013 12:56 PM, Przemyslaw Gorszkowski wrote:
 > 1a. Does Continues Integration system contain automated testing? It
 > would be good to see what is going on with each build.

As far as I know, that's not the case. +1 for my side for this.

 > 1b. Maybe we should also provide some examples, descriptions,
 > information where we can find, how we can create and manage these tests.
 > 1c. There should be also provided some information what kind of
 > tests, frameworks and test suits we currently use/have.

C++ tests
===================

We have two categories of tests:

"Plugin" auto tests (src/plugins/cpptools)
-------------------------------------------
These tests are included in the plugin itself, since they can’t be
separated easily. Currently these are code generation, completion and
model manager tests (files: *_test.cpp).

Run the tests with: $ ./qtcreator -test CppTools

You need a debug build for this (otherwise Qt Creator will not know
"-test").

"External" auto tests (tests/auto/cplusplus)
---------------------------------------------
These are the ‘plugin separable’ tests, based on qttestlib.

Just build and run the relevant test project.

If you’re using shadow builds, make sure to place the build directory
accordingly to the source tree inside the build directory of Qt Creator
since the tests referene libCPlusPlus.so.1 from Qt Creator. If you run
the tests from Qt Creator, you probably want to add the Qt Creator
project as a dependency (Projects, <test project>, Dependencies).

Execute

   $ make check

in order to run to run all tests. If a test fails, subsequent tests will
not be executed, therefore you might want to add “-i” to the command for
ignoring failures and see which other tests also fail.

Creating tests
==============

The tests are based on qttestlib. Just take a look into the doc.

Prefer to add external auto tests, since compared to plugin auto tests
these are faster to run and debug. Apart from this, you can select a
single test to run ("./qtcreator -test CppTools" will execute all tests
of the plugin, which is sometimes annoying.). I haven't yet investigated
on the plugin auto tests so much, so I can't tell you exactly when you
will have to add a plugin auto test instead of a external auto test. Of
course it would be nice if all tests were external auto tests.

Look for currently available tests. Ideally you would just need to add
some further tests cases in a *_data() function. Or you start by
copying/pasting some similar test (and data) and adapt it to your needs.
You may need to refactor a bit in order not to have to much duplicate code.

If there are no similar tests, you will have to set up some kind of
infrastructure / test environment (class/struct).

Managing the tests
==================

Not sure what you mean by "manage these tests". Personally, I've set up
a Session with the relevant projects (qtcreator.pro<http://qtcreator.pro>,
tests/auto/cplusplus/cplusplus.pro<http://cplusplus.pro>, and all the tools). For
qtcreator.pro<http://qtcreator.pro> and tests/auto/cplusplus/cplusplus.pro<http://cplusplus.pro> I've added run
configurations to easily run the tests.

 > 1d. Which parts of editor/code model have some tests and where. For
example:
 >
 >   * code-completion - there are unit tests in cppcompletion_test.cpp
 >   * highlighting (both syntactically and semantically)
 >   * navigation (the locator, follow symbol, etc)
 >   * introspection (the class browser, the outline, etc)
 >   * diagnostics and tool-tips
 >   * find usages and renaming
 >   * refactoring actions - currently there are some unit tests but only
 >     for InsertionPointLocator(cppcodegen_test.cpp)

Most of the stuff in this list is not covered by the tests I've
mentioned above.

However, if I'm not mistaken completely, there are some Squish tests
that might cover some items. Robert Loehning and Christian Stenger (in
CC) might know more about this.

Best regards,
    Nikolai

_______________________________________________
Qt-creator mailing list
Qt-creator at qt-project.org<mailto:Qt-creator at qt-project.org>
http://lists.qt-project.org/mailman/listinfo/qt-creator



--
Best regards/Pozdrawiam

Przemyslaw Gorszkowski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/qt-creator/attachments/20130114/21e68577/attachment.html>