[Development] File Selectors API, still for 5.1?

Aaron J. Seigo aseigo at kde.org
Tue Mar 26 00:23:24 CET 2013

tl;dr -> This is a step in a good direction. I don't think it is usable as-is 
for Plasma, but it could mature into such a thing. I'm unconvinced this is 
ready for 5.1, particularly not as public API, but could be polished into 
something shiny for 5.2

and now I will ramble on below ;)

(as I wrap this email up, it's nearly 00:30, and I have to get up in 6 hours 
.. I wish there were more hours in the day, or less going on at once. ;)

On Friday, March 22, 2013 14:57:11 Alan Alpert wrote:
> Secondly, this is time-sensitive. At least Plasma and BlackBerry
> already have conflicting schemes to provide this functionality on top
> of Qt 4. A disruptive change to their file structures is only going to
> work on a major version change, like when they start using Qt 5 (both
> aim to start with 5.1 I believe). Missing this window means that they
> both continue to use their own conflicting directory structures in
> their next major versions, which is a blow to cross-platform
> application development.

This is quite true. I'd also prefer it if we didn't try and get this Right(tm) 
by rushing it into 5.1 as public API. We can work with private API as long as 
we agree to a schedule as to when the API can change, e.g. only between x.y 
releases ( no API change in 5.1.1, 5.1.2, etc. 5.1->5.2 would be fine)

> 3) Why directories? This change wasn't discussed on the ML yet, and is
> a key difference from the consensus reached in
> http://lists.qt-project.org/pipermail/development/2013-January/009359.html
> .
> In one of the patch sets, the form changed from file at selector.png to
> +selector/file.png . Feedback from designers in BlackBerry suggested
> that they were more comfortable with directories, especially when
> using it to swap out graphical assets for device variants (which can
> lead to a lot of selected files). It's much easier for them to deal
> with a group of selected files via directory, which I presume is
> because from a GUI file manager you don't have the shell globbing that
> I'm accustomed to using. 

This is also what i communicated to Alan by IRC some time ago when the 
consensus was for file name mangling. It is not only that many developers and 
designers find directory based separations easier to manage, but it ensures 
that accidentally naming your file something "special" does not result in 
unexpected behaviour.

I'd suggest that it is quite unexpected to have a file called tiger_800x600.png 
get chosen over tiger.png just because someone thought it was a good idea to 
add "800x600" to the selections.

In Plasma we handle this rather clearly: each package of QML has a contents/ 
directory. In the root of the package, we keep various bits of metadata used 
by the runtime but which are not part of the actual set of resources that make 
up the component in that package. This guarantees no naming collisions between 
whatever special files the runtime may want now or in the future and what 
developers choose to call things.

We then put platform components, or what Alan is called "selected files" (same 
difference :), in a *separate* root, resulting in a package structure that 
looks like this (from an actual example used in production):

├── metadata.desktop
├── contents
│   ├── code
│   │   └── uiproperties.js
│   └── ui
│       ├── Icon.qml
│       ├── menu
│       │   ├── CommentForm.qml
│       │   ├── Confirmation.qml
│       │   ├── MenuArea.qml
│       │   ├── MenuItem.qml
│       │   ├── ServiceMenu.qml
│       │   ├── SlcMenu.qml
│       │   └── TargetChooser.qml
│       └── sharelikeconnect.qml
└── platformcontents
    └── touch
        └── code
            └── uiproperties.js

so platformcomponents/touch/code/uiproperties.js is selected for rather than 
contents/code/uiproperties.js whenever "touch" is one of the selectors and 
"code/uiproperties.js" is requested (well, usually it's in the form: 
file("code", "uiproperties.js"), but that's a matter of a different API, namely 

This strategy ensures no chance of accidental file naming causing an unexpected 
selection or for files required by the runtime to collide with files provided by 
the component developer. It also becomes very clear what are default contents, 
what are selected files, etc. and it is very easy to separate out, remove or 
add new sets of selected files.

Personally, I find the use of "+" in the directory names to be less desirable 
than the above approach, though that is because we do not nest selectors in 

Our typical selector usage has a device specialization, an input paradigm and 
a general form factor, e.g. "WhizzyGoProduct : Touch : Tablet" or "EyeAmaze : 
Desktop". This lends itself to encouraging developers to create common assets 
for generic input and form factor, and only as a finishing touch put product 
specific assets in.

Alan's patches on the other hand encourage an approach based on screen 
resolution and target OS. This is definitely another way of approaching it, but 
is generally an approach we try and discourage in Plasma. Why? Because if you 
are designing for screen resolution, you're doing it wrong. Much more 
interesting is the actual physical size of the device and the input methods, 
as those are the human concerns. I don't care if there are enough pixels on my 
5" device to show your high density UI .. I want something that works well and 
looks proportionate on 5" of glass. In that sense, pixel density is equally 
important ... and then we're off to the races with an NxM matrix of pixel 
density + screen res of directories .. oh, and then add the target OS. :/

Is this really what we want QML development to become? Hyper specialization of 
code bases into as many QML files as devices one hopes to run it on? That's the 
diametrical opposite of the goal of Qt in general.

While it is evident that at least in the near-to-mid term we'll have to deal 
with different platforms having their own QML APIs to offer that are highly 
divergent, it is anything but evident that we ought to base file selection on 
screen resolution.

... and this does not even start to consider that this style of API is 
inherently biased to full screen applications. It is utterly irrelevant to a 
QML application running on a 1280x720 screen if it running in a window or only 
part of the screen.

... or that screen resolutions continue to be variable from vendor to vendor, 
year to year.

So, in summary, all imho from a Plasma perspective:

* selectors are a great concept, and directory based sorting is the way to go
* I'm very much against the addition of screen resolution and DPI as static 
selectors. It essentially encodes the development paradigm in Qt itself in a 
way that we can not get rid of without patching .. and I really don't want to 
have to support developers who think that putting hardcoded screen sizes is a 
good idea (didn't we already live through that on Android?)
* The operating system selectors make sense, but are currently limited and 
provide no way of being extended without patching Qt
* I'm skeptical of the + prefix notation, though I can see how nesting can be 
useful depending on the development paradigms a platform wishes to encourage 
amongst its developers. 

That's the design side ... 

On the implementation side:

* in Plasma, we use a ':' separated list, not a ' ' separated list for the env 
var. This is more in line with traditional path lists on UNIX, and we already 
have this out in production systems.

* I have concerns about the combination of nesting and strict list ordering as 
it can make it complex for the developer to predict the order things will be 
chosen. Consider the asset foo.png and the following directory layout:

├── +A
│   ├── foo.png
│   ├── +B
│   │   ├── foo.png
│   │   ├── +C
│   │   │   ├── foo.png
├── +B
│   ├── +A
│   │   ├── foo.png

For the set of selectors { A, B } +A/+B/foo.png will be selected
For the set of selectors { B, A } +B/+A/foo.png will be selected
For the set of selectors { A, B, C } +A/+B/+C/foo.png will be selected
For the set of selectors { B, A, C } +B/+A/foo.png will be selected

So order of the selectors matters greatly. If one is wanting to match based on 
screen res and platform, then it becomes an absolute requirement that the 
order that those appear is consistent and logical and can *never* be changed. 
This ordering is, essentially, public API.

Currently the order is: env var (random stuff from the platform), screen res, 
locale name (language and country), target platform (none of which are Plasma, 
Sailfish or Ubuntu Phone , of course, though we see Android and Blackberry 

So how do we, as a not-important-enough-to-have-made-it-to-Q_OS_* levels (I'm 
being facetious there ;) ensure our platform is at the same place in the chain 
as others so that the ordering remains the same? Well, we can't. This makes 
this implementation very difficult for us to use and keep the developer story 
straight. I'd rather have the OS moved to right after the env var, or simply 
be required to be set by the runtime itself.

I don't see any reason whatsoever for having the locale name in there. 
Localiation support belongs elsewhere, and most developers will get this wrong 

Whatever the final set of selectors, before this goes into public API, the 
ordering needs to be well defined. Or, alternatively, the selectors must be 
marked up in such a way that the ordering can be done at runtime ... as this 
should only need to be done once the cost would be low to merge multiple sets 
of selectors consistently so that the application, runtime and Qt could all 
provide selectors and have them ordered based on what they are.

This would resolve the issue of inserting a platform as the runtime could 
simple inject sth like "platform: Plasma". heck, it could be a snippet of json 
if we wanted to go all crazylike. this would at least guarantee app developers 
that whatever set of selectors existed, they would be in a consistent 

* Due to the nesting, there are a lot of allocations on every check of the file 
system for a file. There is the concatenated QStringLists in 
QFileSelectors::allSelectors and the creation of new QStringLists in 

* This is really one part of a Package definition; and I'm unsure (because I 
simply haven't sat down and tried to integrate this in any way with libplasma2 
code) if this would work seamlessly with our current concept of a package as 
described earlier.

There is no results caching, but that's OK as that can be done in code that 
uses this, so not a major objection (we already do this in Plasma::Package, 
for instance, to prevent having to constantly hit disk and perform the various 
string and QFileInfo etc allocations in searching for matches)

.. and yes, there is the matter of dynamically changing the set. This is 
something we will have in Plasma in our initial Qt5 port, so it is important 
to us to get that "Right", though that can be done in another round of work 
later. Currently our plan is to have a DBus service that tracks the current 
set of selectors and signals the runtime if/when they change.

This is important for supporting an on-the-fly switch at runtime between, say, 
a tablet form factor and a desktop form factor.

> 4) One question (sort-of) that came up in the review (from Jędrzej):
> "Without runtime detection of a selector value change, the
> QFileSelectors is just a workaround for a deployment problem. It
> solves issues that can be solved by an installer script or even by a
> dynamically loaded QResource." 
> tied to compile-time artifacts. This allows for much faster
> development iteration, as you can just copy graphical assets or data
> to the install dir or device without needing a re-compile. This is a
> valuable feature for most mobile development, and an essential feature
> for interpreted languages like QML.

This is indeed the most important point of all. It is about making developer's 
lives easier and more productive, not trying to solve a problem 
programatically just because we can.

Basically, when developing, one gets to maintain *one* package that they can 
then test and deploy in-situ without first having to run installers that may 
well be platform dependent. Writing such installers to have all the correct 
permutations is error prone and on top of that .. what do you do about 
applications that sit on non-static platforms, where the form factor, screen 
res and other factors may change on the fly? Re-install at runtime in response 
to environment changes? Yeeah.

I also would not worry about performance before we get to measuring it. In 
Plasma, this has not been a bottleneck issue at all and we have not yet even 
gone to such extremes as "put all files into a single file mmap'd read-only 
pseudo-fs" that would probably help with memory fragmentation and access on 
rotating media.

Aaron J. Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.qt-project.org/pipermail/development/attachments/20130326/40c6bc36/attachment.sig>

More information about the Development mailing list