[Development] Module versioning in Qt 5 & the "release manifest"

[henry.haverinen at nokia.com]

Hi all,

We had a discussion (a while back) with Lars, Martin, Aaron, Roberto, Alex
and others about the module versioning in Qt 5: why we have versioned
import statements in QML, how backwards compatibility in C++ and QML can
be maintained, and if there is something we could do to make the module
version numbers a bit easier for the users of Qt. I wrote a summary of
that discussion below.
 
This is partly a clarification on how some basic things work in Qt, and
there is also a proposed tooling and documentation based improvement.
Comments would be of course very welcome :)

Cheers,
Henry 

--

What kind of API compatibility do we promise between minor versions of Qt?

C++ APIs are binary compatible: dynamic linking of old app binaries works
successfully, and there are no incompatible changes in the class
interfaces. 
QML APIs may add symbols (elements, properties, signals, slots) but old
symbols remain unchanged. With versioned import statements, old
applications will be technically able to use the same API (even when it is
provided by a new implementation). In other words, wen though QML has
versioned import statements, we would NOT use them to completely change
the API between minor version releases.
Dynamic behavior is maintained. We must be very mindful so as not to break
applications. Reasonable exceptions can be agreed, when breaks are limited
to few applications and the justifications for the change are good.

In QML, each module has a version number that is shown in the code:

import QtQuick 2.0
import QtContacts 5.0

Because of QML's name resolution rules, symbols that are introduced in
later versions of the module may clash with the application's code, so it
would not be a possible import "the latest" version without causing
compatibility problems. This is the "name shadowing" problem.
The version number in the import statement specifies which elements,
properties, signals, slots the application expects to be exported in the
public API. It solves the name shadowing problem. Another benefit of an
explicit version number is that it also documents to the user of Qt
whether a certain module has changed between Qt releases.

How a new version of an API is implemented?

In C++, it is well known how a new version of an API can be implemented
while maintaining binary compatibility.  Qt releases only include the
latest version of the DLLs.
For QML APIs that are implemented as C++ plugins, it is possible to use
the Q_REVISION and REVISION macros in class definitions to indicate which
symbols have been added in later revisions of the API. Thanks to these
macros, apps that import old versions of the module won't get the new
symbols. For these QML APIs, it is also enough to ship the latest version
of the the library in order to support all the versions of the library.
QML doesn't have anything similar to the Q_REVISION and REVISION macros.
For QML APIs that are implemented in QML, the qmldir file will specify the
mapping from the version number to the QML files that implement the
version. So a Qt release will have to ship all supported versions of the
QML library and a qmldir file. Applications that import an older version
will really get the old implementation of the module. The Qt Graphical
Effects module (that was just blogged in Qt Labs) is an example of a
module implemented completely in QML. Another previous example of QML
implementations is Qt Quick Components for Symbian, MeeGo and desktop.

Since the old versions of QML library implementations need to be included
in the release for compatibility, they will increase the footprint of Qt a
bit. (However, the current examples are small, and if the QML libraries
included larger resources (like images for Qt Quick Components), they
could still be shared between versions.)
The old versions  also need to be tested before the release. In general,
supporting old import statement versions will add some maintenance burden
even for QML APIs that are implemented in C++, because compatibility
should be tested and compatibility bugs related to name shadowing and
other changes should be fixed. It is not always straightforward to get the
 Q_REVISION and REVISION macros right, or to test that they are right, as
we have experienced with Qt Mobility.

What's the problem with explicit version numbers

We expect Qt users to take a certain Qt 5 release, instead of a collection
of hand-picked module versions. Now that each module is versioned
completely independently, developers will have to map their target Qt 5
version to the individual module versions. This imposes a somewhat
unnecessary mental burred to the user of Qt.

Making the versions easier with the "release manifest"

The improvement idea we came up with is to introduce a "release manifest"
file that defines the collection of module versions in a certain Qt
release. The file could list several versions of the same module for the
same Qt release. This idea is building on the current implementation
rather than trying to change anything fundamental. It could be added for
Qt 5.0 or in a later release.

In this proposal, we would include information on the target Qt release
somewhere in the application's project file. The Qt release information
would then map to a certain release manifest file. This information would
mainly be used by tools like Qt Creator to help the developer with the
version numbers in the import statements, and by qdoc (see below). For
example, when you type "import QtQuick " in a project that is targeting Qt
5.0, Qt Creator could auto-complete the statement into "import QtQuick 2.0"

In the future, the manifest could also enable features like import
statement suggestions: if you declare a DropShadow QML element in your QML
file, Creator could figure out that for the targeted Qt release, this
element would require QtGraphicalEffects 1.0 to be imported, and suggest
to add the import statement. Another possible tooling feature would be to
suggest upgrading the import statement version to a later release, if
you're trying to use a symbol that was added in a later version than what
your QML file is now importing. The manifest file could also be used to
check that all import statements are valid against the release manifest of
the targeted Qt release.

In the documentation, the release manifest could be used to provide better
"since" information. Both C++ and QML API reference documentation could
state the Qt release since which the feature has been provided, in
addition to the module version. So the API reference documentation for a
QML element in Qt Graphical Effects could say something like "since Qt
Graphical Effects 1.1 (introduced in Qt 5.2)".