[Development] To improve UX of the online installer

Tue Apr 3 11:27:21 CEST 2018

Thanks for your valuable comments and feedback. Please find my comments online 

> (A) Downloading metadata is very time-consuming.

We have recognized the problem as well and considered several options to solve this issue https://bugreports.qt.io/browse/QTIFW-975. You have excellent points about how to group the repositories. We will take those into account when we plan metadata downloading further. Downloading less metadata, would make the whole download process less vulnerable to network errors. There have been discussions around a retry/resume button and we will evaluate its need after we have evaluated the effect of repository grouping on download time. Obviously, you other suggestions will be considered as well.

> (B) The automatic mirror selection algorithm doesn't always pick the best mirror.

Adding a select/deselect all button in the repositories view or allowing user to choose mirrors, should be rather easy to do. We will consider these. 

> (C) (New users) It's not clear how to choose what to install.

There has been quite a lot of effort in planning and even implementing different approaches to tackle this. Simple default installation would be easy for newbies, but deciding what the default actually means, is challenging. User may have several toolchains installed and it's difficult to decide in the first installation, for which toolchain the user wants to install Qt libraries. This could be asked separately, but in the end the user would have the same component tree view anyway. 

To minimise the number of installable components in a Qt version, we have considered removing the checkbox, so that the user is forced to open Qt X.X node and choose components explicitly. However, we would lose the 'partially checked' feature, used by many. We could maybe implement something that the "Qt 5.9.4" node is not selectable but would still show the 'partially checked' mark.

Showing additional screens before the component view, would move the complexity of component selection from the installer tree view to some screens. We have also received some feedback that the number of screens is already too high. So, we have been considering simplifying the views https://bugreports.qt.io/browse/QTBUG-62671 by using different categories/groups, which define the visible components to the user. This should simplify the first-time installation experience. 

Someone may want to install Qt Creator only, but giving a warning, if now Qt target is chosen is a good idea. 

---
Tino Pyssysalo
Senior Manager
Product management 

The Qt Company 
---

On 03/04/2018, 3.14, "Development on behalf of Sze Howe Koh" <development-bounces+tino.pyssysalo=qt.io at qt-project.org on behalf of szehowe.koh at gmail.com> wrote:

    3 broad issues impede new users who want to start using Qt and
    frustrate old users who want to update Qt:

        (A) Downloading metadata is very time-consuming.
        (B) The automatic mirror selection algorithm doesn't always pick
    the best mirror.
        (C) (New users) It's not clear how to choose what to install.

    ======================================
    Factors that contribute to (A) include:

    * The metadata downloader opens lots of short-duration TCP
    connections. Wireshark revealed 2391 HTTP GET requests from
    77.86.229.90 (download.qt.io) on Windows. This process is less painful
    on Linux (1205 GETs -- almost half of Windows') as there are fewer
    compilers/platforms.

    * Downloading many small files is very inefficient compared to
    downloading a few large files. While GET-ting metadata from
    download.qt.io, my network traffic crawls at ~10 KiB/s (yes, ten
    kibibytes). After this, the actual binaries download from my
    designated mirror at ~1 MiB/s, which shows that metadata downloader
    doesn't make good use of available bandwidth. I'm in Australia.

    * MaintenanceTool downloads metadata too frequently (QTIFW-975).

    * If the metadata download fails halfway (e.g. one file times out),
    the whole process needs to start from scratch. There is no
    retry/resume. This makes the online installer nigh unusable if the
    user's connection to download.qt.io is flaky:
    https://forum.qt.io/post/450076

    * Currently, there is a way to manually avoid downloading unwanted
    metadata: Deselecting the unwanted Temporary Repositories. However,
    all 91 Repositories must be unchecked one-by-one, and this process
    must be repeated each time MaintenanceTool is restarted:
    https://forum.qt.io/post/450076

    ======================================
    Evidence for (B) include:

    * Geographical proximity does not imply mirror quality. E.g. a user
    from China finds Chinese mirrors too slow, and opts for non-Chinese
    mirrors instead:
    http://lists.qt-project.org/pipermail/interest/2013-December/010336.html

    * The algorithm doesn't pick the closest mirror anyway. E.g. a user
    from Israel was given a Japanese mirror instead of a European mirror:
    https://forum.qt.io/topic/87221/

    ======================================
    Examples of (C) include:

    * Some users don't realize there are multiple kits per Qt version.
    They select "Qt 5.9.4" and then wonder why Qt is huge (35 GB):
    https://forum.qt.io/topic/87608/windows-install-download-size

    * Some users don't realize Qt Creator != Qt. They install Qt Creator
    without Qt and end up unable to build anything:
    https://forum.qt.io/topic/84198/no-valid-kits-found

    * Some users don't realize they need to install a compiler separately.
    They also end up unable to build anything:
    https://forum.qt.io/topic/79532/msvc2015-64bit-compiler-kit-installation-requirements

    ======================================
    Ideas for improvement (some render others unnecessary):

    1. Amalgamate and compress all metadata into one file (or at least one
    file per "group").

    2. Cache metadata locally and use a hash to identify them; avoid
    re-downloading metadata that's already available (Mitch Curtis
    suggested this at QTIFW-975)

    3. Avoid downloading metadata for archived/advanced packages by
    default (Iikka Eklund said this is planned in QTIFW-975)

    4a. Remember the which Repositories the user has deselected before,
    and ignore them for future sessions.

    4b. Let the user choose broad "groups" that they are interested in,
    and remember this choice. For example, there is no point getting
    metadata for Android/UWP packages or showing these packages in the
    "Selection Tree" if I'm only interested in Desktop development.

    5. Add a "Select/Deselect All" button to the Repositories page.

    6. Allow retry/resume if the metadata download fails halfway.

    7. Allow users to manually pick a mirror for binary download.

    8a. Before the user is asked to "Select the components to install",
    show a short page that to explain the difference between Qt and Qt
    Creator and explain that a compiler must be installed separately
    unless MinGW is used.

    8b. Before the user is asked to "Select the components to install",
    have a small wizard to ask a new user some questions -- Which
    version(s): Latest version, LTS version, or older version? Which
    target platforms? Which compiler? Or do they not care/know and want us
    to suggest a good default? Use the answers to pre-fill the "Selection
    Tree" and remind the user to download an external compiler if
    necessary.

    9. If no Qt version is selected (or if too many are selected), pop-up
    a confirmation dialog before installing.

    10. Expand the documentation at http://doc.qt.io/qt-5/gettingstarted.html

    Since (3) is already being planned, please consider (4) which is a
    generalization of (3).

    It would be fantastic to see a holistic approach that addresses (A) +
    (B) + (C). For example, (8b) helps the user make the right selection
    at the first try. At the same time, users' answers in (8b) feed into
    (4b) to download fewer metadata files and trim the "Selection Tree".
    This is further sped up by (1) + (2): Each "group" has its own
    amalgamated metadata file, and these files are cached locally to avoid
    re-downloading in the future. Once the selection is complete, (7)
    ensures that the use can get binaries in a timely manner even if the
    auto-selected mirror is no good.

    Regards,
    Sze-Howe
    _______________________________________________
    Development mailing list
    Development at qt-project.org
    http://lists.qt-project.org/mailman/listinfo/development