[Qt-creator] Editors/Code-models: new year's resolutions

[Nikolai Kosjar]

Hi!

On 01/12/2013 12:56 PM, Przemyslaw Gorszkowski wrote:
 > 1a. Does Continues Integration system contain automated testing? It
 > would be good to see what is going on with each build.

As far as I know, that's not the case. +1 for my side for this.

 > 1b. Maybe we should also provide some examples, descriptions,
 > information where we can find, how we can create and manage these tests.
 > 1c. There should be also provided some information what kind of
 > tests, frameworks and test suits we currently use/have.

C++ tests
===================

We have two categories of tests:

"Plugin" auto tests (src/plugins/cpptools)
-------------------------------------------
These tests are included in the plugin itself, since they can’t be 
separated easily. Currently these are code generation, completion and 
model manager tests (files: *_test.cpp).

Run the tests with: $ ./qtcreator -test CppTools

You need a debug build for this (otherwise Qt Creator will not know 
"-test").

"External" auto tests (tests/auto/cplusplus)
---------------------------------------------
These are the ‘plugin separable’ tests, based on qttestlib.

Just build and run the relevant test project.

If you’re using shadow builds, make sure to place the build directory 
accordingly to the source tree inside the build directory of Qt Creator 
since the tests referene libCPlusPlus.so.1 from Qt Creator. If you run 
the tests from Qt Creator, you probably want to add the Qt Creator 
project as a dependency (Projects, <test project>, Dependencies).

Execute

   $ make check

in order to run to run all tests. If a test fails, subsequent tests will 
not be executed, therefore you might want to add “-i” to the command for 
ignoring failures and see which other tests also fail.

Creating tests
==============

The tests are based on qttestlib. Just take a look into the doc.

Prefer to add external auto tests, since compared to plugin auto tests 
these are faster to run and debug. Apart from this, you can select a 
single test to run ("./qtcreator -test CppTools" will execute all tests 
of the plugin, which is sometimes annoying.). I haven't yet investigated 
on the plugin auto tests so much, so I can't tell you exactly when you 
will have to add a plugin auto test instead of a external auto test. Of 
course it would be nice if all tests were external auto tests.

Look for currently available tests. Ideally you would just need to add 
some further tests cases in a *_data() function. Or you start by 
copying/pasting some similar test (and data) and adapt it to your needs. 
You may need to refactor a bit in order not to have to much duplicate code.

If there are no similar tests, you will have to set up some kind of 
infrastructure / test environment (class/struct).

Managing the tests
==================

Not sure what you mean by "manage these tests". Personally, I've set up 
a Session with the relevant projects (qtcreator.pro, 
tests/auto/cplusplus/cplusplus.pro, and all the tools). For 
qtcreator.pro and tests/auto/cplusplus/cplusplus.pro I've added run 
configurations to easily run the tests.

 > 1d. Which parts of editor/code model have some tests and where. For 
example:
 >
 >   * code-completion - there are unit tests in cppcompletion_test.cpp
 >   * highlighting (both syntactically and semantically)
 >   * navigation (the locator, follow symbol, etc)
 >   * introspection (the class browser, the outline, etc)
 >   * diagnostics and tool-tips
 >   * find usages and renaming
 >   * refactoring actions - currently there are some unit tests but only
 >     for InsertionPointLocator(cppcodegen_test.cpp)

Most of the stuff in this list is not covered by the tests I've 
mentioned above.

However, if I'm not mistaken completely, there are some Squish tests 
that might cover some items. Robert Loehning and Christian Stenger (in 
CC) might know more about this.

Best regards,
    Nikolai