[Development] minicoin (was: Configure command lines of official Qt releases)

[Volker Hilsheimer]

Taking the liberty to starting a new thread for the minicoin related discussion...

On 4th of June 2019 at 6:09 PM, Konstantin Tokarev <annulen at yandex.ru> wrote:
> 04.06.2019, 17:01, "Volker Hilsheimer" <volker.hilsheimer at qt.io>:
>>> 04.06.2019, 16:41, "Volker Hilsheimer" <volker.hilsheimer at qt.io>:
>>>>> "Why don’t we make the exact way of turning a clean Linux
>>>>> distro-install into a "Qt reference configuration" available to
>>>>> everyone else? The way build machines are provisioned in Coin is
>>>>> rather opaque, even with some of the respective provisioning scripts
>>>>> available in the qt5.git repo [1]. Having to document on a
>>>>> (notoriously outdated) wiki how to set up things to build Qt from
>>>>> source, when we have that knowledge literally codified somewhere for
>>>>> Coin, doesn’t seem effective."
>>>> 
>>>> Since I’m being quoted, I might just as well use the opportunity to announce that I’ve just made the repo where I’ve been tinkinering on a solution to this problem for a while now public on the Qt gitlab instance, accompanied by a little blog post:
>>>> 
>>>>  https://blog.qt.io/blog/2019/06/04/introducing-minicoin/
>>> 
>>> Can this project be used to test provisioning changes before submitting them to the "big" Coin,
>>> or there may be behavior differences?
>>> 
>>> Either way, nice job!
>> 
>> Thanks!
>> 
>> Testing provisioning scripts and making sure that they result in a working setup is definitely one of the use cases for minicoin. I found it very useful to be able to have a fast loop of “minicoin up/try to build stuff/minicoin destroy/improve script”, e.g when trying to build Qt for Android locally.
>> 
>> There are however some key differences in the assumptions Coin and minicoin make; for example, Coin has a few helper scripts that are used to download version-pinned packages from a Qt-internal cache, which is the kind of stuff minicoin doesn’t care about, so scripts at this point don’t translate 1:1.
>> 
>> But If you run “minicoin status” you see that a bunch of coin-* machines are defined, which will run the provisioning scripts from qt5.git/coin/provisioning. So there is some basic scaffolding, and perhaps stuff that lives in the qt5.git open source repo should be useful by anyone, even if they don’t run their code within The Qt Company network :)​
> 
> I guess it won't be possible to obtain some OS images like Windows or macOS outside of The Qt Company network because of licensing issues, however it would be great to have at least Linux and Android.

Indeed, we can't make the Windows or macOS images publicly accessible, which is a shame since a lot of work and yak-shaving went into building and packaging those boxes.

FWIW, for macOS images I just used the ones you can download from AppStore; general Apple-restrictions about virtualising macOS apply anyway. For Windows, there are a few images on the Vagrant cloud, and I might try to add some of the setting-up scripts to the "basebox” repo. One challenge is as well that those OS images are huge.

For Linux - you can just use what's publicly available on the Vagrant cloud. The stuff in the "generic" namespace is very well maintained, and packaged using Packer with all source code available here: https://github.com/lavabit/robox

You can be pretty certain that you are not downloading some malicious malware in a black box to your system. These are all based on server-images, which makes them fairly small and lean; installing a Linux desktop and X11 or Wayland server etc should be done when provisioning, using roles (I’ve started with a generic linux-desktop role). That way you don't have to download a lot of prepackaged GBs of software that you often don't need anyway.

For Android - to build Qt for Android, I'm using generic/ubuntu1804 and install the SDK/NDK toolchain. See the "android-builder" box definition:

https://git.qt.io/vohilshe/minicoin/blob/master/minicoin/boxes.yml#L76
https://git.qt.io/vohilshe/minicoin/blob/master/minicoin/roles/android-builder/provision.sh


For running things on Android, the idea is then to use the emulator within that VM; that doesn't work with VirtualBox, as it doesn't support nested virtualisation; hence the VMware provider is preferred for this box. This is still work in progress.

I might misunderstand your request though...

> BTW, what about supporting KVM? Unlike VirtualBox, it supports paravirtualized disk I/O which should make very positive impact on build times.

I have not tested anything with other Vagrant providers than VirtualBox or VMware yet, but there are provider plugins for qemu, libvirt, and kvm. Box availability will vary, but the generic/ boxes are available for libvirt at least:

https://app.vagrantup.com/boxes/search?provider=libvirt&q=generic

The multi-machine Vagrantfile used by minicoin uses provider specific configurations and stuff to implement some features, so you'd need to add relevant sections for other providers, ie. if you need to configure RAM or insert virtual disk or hard drives. I might give this a shot later, if I manage to set up kvm etc on my Mac (perhaps we need a meta-minicoin?).


Anyway, thanks for trying the stuff out!

Cheers,
Volker