[Development] Proposal: move Qt provisioning scripts and 3rd party components into a dedicated repo

[Laszlo Agocs]

Hi,

> Our 3rd party dependencies currently live in the submodules where they are used. For some 3rd party components, that means we have two, sometimes different copies (e.g. assimp in both Qt Quick 3D and Qt 3D, only one of them patched). And it makes it in general messy to maintain an overview of those 3rd party components. We have a responsibility to keep track of those, and to keep things up-to-date. I think that would be easier if we move them into a single repo with all 3rd party libraries.

I can try sharing some concerns, many of which are similar to those voiced in the discussions we had around and pre-6.0, when there was a lot of talk about package management and such for 3rd party dependencies (which then died off eventually).

1. Can we assume every single 3rd party library out there has a sophisticated build system that builds (or at least allows building after appropriate configuration) exactly the way Qt needs, with first class cross-compilation and embedded support, including the exotic targets like INTEGRITY? (unlikely)  How about Yocto where you wouldn't need the upstream build system but rather appropriate recipes for packages? Integrating the external code into Qt's build system have some benefits that are probably not obvious at first glance, esp. when it comes to the less straightforward non-desktopish targets.

If the end result here is a partial solution for the common desktop platforms, while most Qt modules will still be stuck with having to support alternative solutions (e.g. still bundle) for the more exotic (but essential) targets, then that might not be much of an improvement over all.

2. Not every 3rd party library offers Qt-style compatibility guarantees. We may not even have proper source/behaviorial compatiblity promises in some cases. This means blindly updating stuff to the latest version (no matter how important or great that release may seem) or sharing the same 3rd party content between multiple Qt modules is not always feasible or wise.

A 3rd party library is often an integral part of a module, with custom patches applied and *careful testing* with every single update of the 3rd party code to make sure it plays perfectly together with the particular module.  (may not be the case everywhere in Qt, but certainly true for some of the graphics/3D dependencies)

Dumping the full upstream contents to a faraway external repository and claiming it will just work as-is (or that any change we apply on top is fine for all clients of the 3rd party lib within Qt) sounds a bit optimistic. The 3rd party content that is essential for a Qt module should rather be "near the eyes of the developers", ideally next to the module sources. (again, this may not apply uniformly true within Qt, but certainly true in graphics/3D for certain 3rd party deps that provide core and essential functionality) 

3. Many 3rd party components are stripped down, meaning we build only what is needed by Qt. In the form that is best, even if that is something the upstream build system would never do, with settings or changes upstream may not support out of the box. (e.g. sometimes we can just compile in the 3rd party sources directly to the module or maybe use a static lib etc., instead of polluting everything with tens of shared libraries and unnecessary tools and artifacts from the upstream build system)

That this makes updating more difficult is not a valid argument, since updating *should* be an involved process - the impact of updating the 3rd party code must be carefully evaluated and tested. (as always, thinking of graphics/3D 3rd party examples here, perhaps the bar is lower elsewhere with other 3rd party deps)

4. Simplicity of customization. Applying custom patches to a stripped down bundled version may be significantly simpler than fighting the entire upstream repo and its build system. (and customization will be needed, given Qt's extensive platform and compiler coverage)

5. Manually building and installing all 3rd party dependencies for every single Qt module we work with (thinking here of devs with no toplevel builds and whatnot, just building individual repos) is not very tempting.

Best regards,
Laszlo



-----Original Message-----
From: Development <development-bounces at qt-project.org> On Behalf Of Alexandru Croitor
Sent: Thursday, July 14, 2022 11:53 AM
To: Volker Hilsheimer <volker.hilsheimer at qt.io>
Cc: development at qt-project.org
Subject: Re: [Development] Proposal: move Qt provisioning scripts and 3rd party components into a dedicated repo

Hi,

Just pointing to some relevant issues and discussions.

https://bugreports.qt.io/browse/QTBUG-68816
https://bugreports.qt.io/browse/QTBUG-73760
https://wiki.qt.io/QtCS2018_Third-Party_Sources_Policy_and_Security

> For us as Qt developers it might mean that we either have to build that 3rd party repository ourselves (which could still be done as part of the toplevel build process)

If the libraries will be built using their native build system instead of CMake, that will complicate the implementation of top-level builds, as well considerably slow down configuration time, because the 3rd party libs will need to be built and installed in some staging area before CMake can pick them up for Qt usage at configure time.
So qt5/configure would imply first running configure && make && make install for each 3rd party library.
_______________________________________________
Development mailing list
Development at qt-project.org
https://lists.qt-project.org/listinfo/development