[Development] Building additional components with Conan for Qt 6

[Toni Saario]

>>>  1. Will Conan be used to manage dependencies of Qt as well?
>>>
>>> If this is the plan, note that there is infrastructure [1] for using Conan in provisioning images,
>>> with verification of downloaded file checksums and build reproducibility for each qt5 commit [2].
>
> Regarding Qt CI @Toni Saario is the best one to answer on this, though I haven't heard
> if there are plans to use Conan more extensively by CI/provisioning at this point?

>From my experience, when the way project is built in CI is radically different from a way suggested to
the end user, it inevitably leads to troubles. Users hit unexpected errors (in this case they may be
caused by changes in 3rd party Conan recipes/packages, or by contributors to Qt module which
don't use Conan), and they come back to me asking for solution.

As Iikka said that there has not been plans to use Conan when provisioning the images for CI.

However there has been some discussions how to make the provisioning easier to manage and have some documentability in them. Currently the provisioning is just a pile of scripts which have zero value for documentation tools. Easiest solution for this would be to list the dependencies on some other file in common format and then install listed dependencies with simple scripts. About using Conan there, I cannot really say anything about it with my knowledge about it.

Another thing is that there are many items built during provisioning which become dependencies, can we reduce the provisioning workload by having these cached in Conan storage? One thing to check is that how much the provisioning it is possible to translate to Conan, I would believe that in the end it does not work for everything.

As for provisioning and why the CI images are different from what are documented, biggest problem there is that the images are used by other components also e.g. RTA testing. Those require some additional stuff and provisioning currently installs everything. AFAIK also all the optional dependencies are installed, this makes it hard to distinguish between them and required dependencies because both are always present. These make it easy that new dependencies etc. slip through unnoticed.

-Toni
________________________________
From: Konstantin Tokarev <annulen at yandex.ru>
Sent: Friday, October 2, 2020 3:20 PM
To: Iikka Eklund <iikka.eklund at qt.io>; Richard Weickelt <richard at weickelt.de>; development at qt-project.org <development at qt-project.org>; Toni Saario <toni.saario at qt.io>
Subject: Re: [Development] Building additional components with Conan for Qt 6



02.10.2020, 14:47, "Iikka Eklund" <iikka.eklund at qt.io>:
> Hi Konstantin, all,
>
>>>  thanks for the heads up. I have some further questions:
>>>
>>>  1. Will Conan be used to manage dependencies of Qt as well?
>>>
>>> If this is the plan, note that there is infrastructure [1] for using Conan in provisioning images,
>>> with verification of downloaded file checksums and build reproducibility for each qt5 commit [2].
>
> Regarding Qt CI @Toni Saario is the best one to answer on this, though I haven't heard
> if there are plans to use Conan more extensively by CI/provisioning at this point?

>From my experience, when the way project is built in CI is radically different from a way suggested to
the end user, it inevitably leads to troubles. Users hit unexpected errors (in this case they may be
caused by changes in 3rd party Conan recipes/packages, or by contributors to Qt module which
don't use Conan), and they come back to me asking for solution.

>
> What comes to add-ons (https://doc-snapshots.qt.io/qt6-dev/qtmodules.html#qt-add-ons) as mentioned
> earlier the idea is to release add-ons as src packages accompanied with conanfile.py i.e. the build recipe.
> If the add-on depends on 3rd party libraries which are available e.g. in conan-center the recipe could
> naturally utilize packages from there (openssl, zlib, etc.).
>
>>     2. How will the recipes be managed and where on code.qt.io can I find them?
>
> The recipes of the first two add-ons are not part of the source repositories yet, probably should be.
> Though there are some drawbacks in keeping the recipes and sources in the same repo
> (some related discussion https://github.com/conan-io/conan/issues/4734).
>
> Currently the recipes are part of an internal packaging repository but the current recipes can be found from the src
> installation directory: "6.0.0/Src/qtnetworkauth/conanfile.py" after you have installed an add-on package.
>
> Iikka Eklund
> Senior Software Engineer
>
> iikka.eklund at qt.io
> http://qt.io
>
> ----------------------------------------
> From: Konstantin Tokarev <annulen at yandex.ru>
> Sent: Friday, October 2, 2020 12:15 PM
> To: Richard Weickelt <richard at weickelt.de>; Iikka Eklund <iikka.eklund at qt.io>; development at qt-project.org <development at qt-project.org>
> Subject: Spam: Re: [Development] Building additional components with Conan for Qt 6
>
> 01.10.2020, 16:16, "Konstantin Tokarev" <annulen at yandex.ru>:
>> 01.10.2020, 16:07, "Richard Weickelt" <richard at weickelt.de>:
>>>  Hello Ilkka,
>>>
>>>  thanks for the heads up. I have some further questions:
>>>
>>>  1. Will Conan be used to manage dependencies of Qt as well?
>>
>> If this is the plan, note that there is infrastructure [1] for using Conan in provisioning images,
>> with verification of downloaded file checksums and build reproducibility for each qt5 commit [2].
>>
>> If you are interested in details we could discuss them in depth in IRC.
>>
>> [1] https://code.qt.io/cgit/qt/qt5.git/tree/coin/provisioning/common/windows/conan.ps1
>> [2] requires https://codereview.qt-project.org/c/qt/qt5/+/313930 to handle situations when the same package version is rebuilt without version bump
>
> Iikka, any comments?
>
> --
> Regards,
> Konstantin


--
Regards,
Konstantin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20201002/8cf5c9f6/attachment-0001.html>