[Qt-creator] Unit testing private parts of shared libraries

[Nikolai Kosjar]

On 10/16/2017 06:57 PM, Elvis Stansvik wrote:
> Hi all,
> 
> (Posting this as a new thread instead of necroposting to my old thread
> about the design of Qt Creator [1], which did end with some
> discussions about testing.)
> 
> I'm working on some tests for my own application, and started thinking
> about unit testing of private parts of shared libraries (think the
> _p.h/_p.cpp parts). Such parts are normally not exported AFAIK (in the
> visibility attribute sense). This becomes a problem if you link the
> unit tests dynamically against the code under test, as the tests won't
> be able to see those symbols.
> 
> So it seems to me there is only a few options:
> 
>   1. Exporting the private parts after all,
>   2. Adding the sources of the code under test to the unit tests,
>   3. ?
> 
> 1 is not really good, as you export more than you really have to. But
> I guess it's not so bad after all, since you a) have to include _p.h
> to get at them and b) they're normally called PrivateSomething or
> InternalSomething, both of which should tell you you're doing
> something bad. 2 is not really good either though, as you really want
> to test the code as shipped, not as compiled as part of your test.

Note that while "test the code as shipped" is very valuable, it does not 
scale very well (if used exclusively) - in the long run it can take 
quite some time to run those tests. If they take too long, you will less 
often run them as you probably should (or they can help you) and forget 
it at times. Eventually you might find they even stay in your way :>

That's why some proposed definitions of "unit testing" go to the extreme 
and use test doubles (fake, stub, mock...) where possible. "test the 
code as shipped"-tests are then considered as integration tests and are 
in quantity significantly less then those light-weight "mocked" unit 
tests. Downside with those unit tests are the needed extra 
interfaces/templates and what not which can make the real code look more 
complicated/abstract than actually necessary.

> I had a quick look at Qt Creator because I know you folks are seasoned
> developers, and it looks like you've gone with option 1. For example,
> ExtensionSystem::Internal::PluginSpecPrivate is exported, and I guess
> the reason is you want to be able to unit test it?
> 
> I just wanted to ask in case you have any other ideas or better ways
> to let unit tests access things that are normally not exported?
> 
> Thanks in advance,
> Elvis
> 
> [1] http://lists.qt-project.org/pipermail/qt-creator/2017-September/006712.html
> _______________________________________________
> Qt-creator mailing list
> Qt-creator at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/qt-creator
>