[Development] status and use cases for QScreen

[Shawn Rutledge]

I got started working on QScreen, its properties, notifiers, and
implementation on all 3 platforms after I noticed that the
documentation was out of sync with the implementation in Qt5.  It
turned out to be more work than I thought.  ;-)  These patches have
gone in already:

beab941e1fc11c8fe0914c6a3207ba029eb96112 X11 (xcb) support for
QScreen-per-output and runtime changes
f4c1ae672652bc908ea1e0a8e353557b392940d7 Added manual test for QScreen
properties
1a31561178d9cb9e5a6f3f986075df24ea5705ff xcb: dynamic QScreens;
primary first; corrected logical DPI
9cec2def90c9644cc75b26d3c9bd38739b9307da xcb: better error handling if
no randr or screen resources query fails
e424354e85671928e6a1cf11cb5cf0a89ba9357f xcb: consistently check
xcb_get_extension_data for extension inits
d0573aa914e59c2947fabb6960735981286cf772 osx: support for QScreen
changes at runtime, and physical dimensions

and these are pending CI:

Windows QScreen: handle change in working area when the taskbar moves
Change-Id: Ic496419d89b9cca7653e77ef22978597abcb4e30

QScreen's physicalSize can change.  Ensure siblings are updated.
Change-Id: I6a98a1c203e24e8fbeef6e90999671e56dc0c655

The current status is
1) you can iterate the QScreens to find out which screens are active
2) each screen has lots of properties like physical size (mm), pixel
geometry (QRect, so that it has an offset w.r.t. the virtual desktop),
virtual size (whole desktop), logical DPI (how big is a "logical
pixel"), physical DPI, etc.
3) those properties can all be updated dynamically when you change
configuration of your screens
4) QScreens can be created and destroyed when you add or remove a
monitor.  You can connect to QObject::destroyed and
QGuiApplication::screenAdded to be notified.
5) there is a test application in qtbase/tests/manual/qscreen which
just maintains a property list window on each screen, and you will see
it update when you change configuration

If the monitor DPI is reported wrong, or the logical DPI is something
different than what you like, you will have to override it.  But
there's nothing new about that.  E.g. you can run startx -- -dpi 96 to
fix the logical DPI at 96.  Qt will use logical DPI to decide the size
of a "logical pixel", which determines the size of fonts, icons etc.
Likewise in most control panel applications in most desktop
environments (KDE, Gnome, Windows, but not OSX) you can override the
DPI.  This is a good way to "zoom" everything on the screen at once,
because most GUI toolkits use the logical DPI.  So I think it's an
improvement that Qt5 is behaving more like the other toolkits now in
this regard.  But I do have one monitor at work which doesn't report
its size correctly, so if I don't override the DPI, then one screen is
around 96 DPI and the other appears to be around 300.  X and Qt both
agree, that the average is something like 182 DPI horizontal and 102
vertical, for an average of around 142.  This results in widgets being
scaled incorrectly, as if the screen was really 142 DPI.  I'm not sure
what to do about that other than just override the DPI.  But for some
reason, non-Qt apps aren't behaving particularly badly in this
situation.  And I don't want to do any quick fixes which will have
negative consequences when HiDPI screens are more in use (it's just
around the corner).  We even need to handle moving windows between
screens, some of which might have very different resolutions.  So I
don't really consider it a Qt bug to deal with the consequences of a
misbehaving monitor, especially since there is a workaround.

Another issue is that on Mac OS X, I used the API available to detect
"mirror sets" (the case when you are showing the same desktop on both
your "main" screen and a secondary one), and to detect which one is
the primary of the mirror set.  I thought it might be less confusing
and bug-prone to just ignore the non-primary display, because why
would an application care about a screen which is just a copy of the
primary one, and doesn't add anything to the virtual desktop?  But
today I realized that making it behave the same on Linux is harder,
because you can use xrandr to arrange screens so they are
side-by-side, partially overlapping, or fully overlapping.  So one
screen will not necessarily be forced to be an exact mirror of
another, and therefore I cannot say for sure which one is the primary
and which one ought to be ignored.  So you will always get two QScreen
instances in that case, and that has me thinking maybe it should be
the same on OSX.  You would need to pay attention to which one is
first in the list of virtualSiblings; according to the docs, the
primary is guaranteed to come first.  But what if you have 3 screens:
2 are mirrored and one is not?  Then maybe you need to pay attention
to the geometry to decide which parts of which screens overlap which
others.  xrandr is so flexible that it's hard to make assumptions.

Anyway the main purpose of starting this thread is to talk about use
cases for this stuff.  There are two I can think of: the most common
might be if you want to write a presentation tool with Qt, you might
want to know stuff about the screen in the presenter's laptop and
whatever type of large screen is showing the presentation.  They might
have different geometry (until recently, high-res projectors were too
expensive, or you might even be using a composite TV output in the
worst case).  They might be mirrored so that you can see what the
audience is seeing, without having to turn around and look; or they
might be separate, so that you can show a full-screen slide for the
audience and use the laptop screen to see a slide sorter, to take
notes, and otherwise plan and control the presentation.  I don't do
much of this myself, so I'm not sure which way is most preferred at
this time.

The second use case I can think of is when you write an application
that has multiple windows, with content in one window and floating (or
dockable) windows for controls etc.  For example, the Gimp.  Suppose
you have a Cintiq for the drawing, and a touchscreen which you would
like to use for the rest of the controls (the toolbox, layers dialog
and so on).  As long as you have a mouse, and both screens are part of
the same virtual desktop, you can arrange the windows that way, so the
Gimp doesn't really need to be aware, but maybe some applications
would want to do this kind of smart layout to spare the user the need
to do it manually.  Perhaps a vertical-market studio app for print
layout or animation or movie editing.  Also, if you have a Cintiq you
really want to map the pen to just cover the area of the screen
itself, pixel-for-pixel.  X11 by default will map it like a mouse,
covering the whole virtual desktop, so small movements of the pen
result in large movements of the cursor.  This can be fixed.  I have
an early-model XGA cintiq at home, and managed to configure it so that
the pen can cover just the Cintiq while the mouse can go anywhere.
But I wonder if it would be useful for Qt apps to be able to get
information about which input devices are available on which screens.
If there was metadata for an input device analogous to that for
QScreen, with a QRect geometry property, maybe that would be enough.
Or maybe the QScreen ought to have a list of input devices which can
go into that screen's space.  The need for this may become more
ubiquitous with tablets: again you might write a presentation app for
a tablet, which might have an HDMI output, but if the touchscreen is
the only input device, then you do not want to put any interactive
controls onto the HDMI screen.  But whether we can get this kind of
information on every supported OS is questionable.

What other use cases do you have in mind for this stuff?