[PySide] Semantic Versioning

[Cristián Maureira-Fredes]

On 10/18/22 22:26, icfwm at gmx.net wrote:
> Hello,
> 
> I'd like to bring up a topic to discussion which I know the QT project 
> takes very serious and I'd like to know if there are any plans to 
> implement this also on PySide6 and shiboken6.
> 
> Semantic versioning seems to be a widely adopted standard in many 
> projects (for details, see https://semver.org/). It basically means that 
> a library maintains backward compatibility when changing the minor 
> version number and it maintains forward/backward compatibility when 
> changing the patch version number (this is usually used for bugfix 
> releases).
> 
> In (C++) QT, the rules are even more strict, because QT guarantees also 
> binary backward compatibility when increasing the minor version number. 
> This basically means that it should be possible to use QT shared objects 
> 6.4.0 for a project originally compiled against 6.0.0 
> (https://wiki.qt.io/Qt-Version-Compatibility).
> 
> In PySide / Qt for Python the version number seems not to give a lot of 
> guarantees. For example, the enum change introduced in 6.4.0 (with the 
> preview introduced in 6.3.1) doesn't seem to maintain backward 
> compatibility (a project written for 6.0.0 doesn't work anymore with 
> 6.4.0). Admittetly, the "forgiveness mode" solves some (or most) of the 
> issues, but the blog post does mention cases where this is not going to 
> work. And I have found also a few cases which broke (see my previous 
> message). Obviously, the environment variable approach with 
> PYSIDE63_OPTION_PYTHON_ENUM is not maintaining backward compatibility 
> either. Therefore, a project respecting semantic versioning rules would 
> have needed to change the major version number for this change.
> 
> The situation gets even more unclear when shiboken comes into the game 
> additionally. When you have a project which uses shiboken to generate 
> python bindings for c++ classes (which additionally depend on C++ QT), 
> even switching between patch numbers has been binary incompatible very 
> often. I didn't try with 6.x.x versions, but I assume that this is still 
> the case? This is a pity because this basically means that the PySide6 
> dependency of such a project is a strict "PySide6==6.minor.patch" 
> because PySide6 has a dependency to "shiboken6==6.minor.patch". I'd love 
> to see at least binary compatibility between the patch versions, but 
> this would require the shiboken API to be binary compatible, which was 
> not the case in the 5.x.x versions.
> 
> So the question is: Are we going to see more backward compatibility in 
> the future of PySide6 / shiboken6? Will there ever be any guarantees?
> 
> Kind Regards
> Christoph

Hello,

thanks for your interest.

# The short answer:
The only compatibility we can provide in this situation
is within minor releases, we cannot include new features in major
releases only because then due to the slow development cycle in Qt,
we would be tied to old Python version for too long.

# The long answer:
We are in a very special position with the Qt for Python project,
where we are trying to find a middle ground of what does the Qt project
and C++ community needs/wants/uses, and what the Python community
needs/wants/uses.

Initially, the first limitation was to provide the module
for the many Python versions, including back then, support for Python 
2.7 and 3.x. We were tied to Python 2.7 for more than a year
after it was deprecated because we didn't want to drop support on
a minor version, which is sane, many people sadly were still on that
version, so we made the decision to drop Python 2 support in Qt 6
releases.
In the meantime, there were Python minor releases that were adding
new features that users wanted to use, but we couldn't because we
decided for a long time to adhere to the Python Limited API [1];
in case you are not familiar with it is just a set of functions
that are guaranteed to work from Python 3.6+, so the pro
is that we need only 1 Python package (wheel) for Python 3.X,
instead of one per-version. The cons is that we cannot use
new features/API internally.

With that information, let's jump into the Enum change.
Due to the compatibility between Python 2 and 3,
Shiboken implemented an ad-hoc Enum system to mimic Qt enums,
and bring them to Python, the main motivation was that
Python didn't have any Enum mechanism in the 2 version,
so we used that, and kept it around. But Python
introduced the enum module in 3.4, so we really wanted
to avoid re-implementing the wheel (no pun intended) and having
our own Enum scheme was not optimal, so the work to migrate
to the Python enum system, started.
Additional motivation here was that PyQt, the set of bindings
developed by Riverbank, was also heavily using the enum module
for Qt 6 related wheels - we wanted to ensure people could
move around PySide and PyQt without much trouble.
So we did the release of the feature in 6.3.1 so people could
be able to test, and have a smoother transition to 6.4,
which by many reasons depending of the reality of developers,
they are unable to do, so that's the motivation for the forgiveness
mode. Yes, there are cases where it will not solve the issues completely
because it was not possible to handle the 100% of the cases due
to the implementation details.

Now we need to jump into Python's evolution.
The moment the firsts alpha, beta, releases of 3.11
were out, we had users starting to ask if it was supported,
or reporting issues related to building it. Even today, Python 3.11 is 
not released, and we managed to provide support for the last RC.
So on the same position, which your valid concern about stability,
we have people asking for supporting the latest and greatest,
which sadly cannot be implemented easily, because even Python
is changing internal mechanism quite often in the minor releases.
(If you are familiar to Python development, you can notice that
there are no plans for a Python 4, and most of the new features
does come in minor releases)

I hope the email was not too long, but this is certainly
a problem that has many variables to consider.


Cheers

[1] https://docs.python.org/3/c-api/stable.html


-- 
Dr. Cristian Maureira-Fredes
Senior R&D Manager

The Qt Company GmbH
Erich-Thilo-Str. 10
D-12489 Berlin

Geschäftsführer: Mika Pälsi,
Juha Varelius, Mika Harjuaho
Sitz der Gesellschaft: Berlin,
Registergericht: Amtsgericht
Charlottenburg, HRB 144331 B
--