[Development] Input events handling ideas and feedback

Shawn Rutledge Shawn.Rutledge at qt.io
Mon Jun 27 10:39:42 CEST 2016

> On 25 Jun 2016, at 15:15, Filippo Cucchetto <filippocucchetto at gmail.com> wrote:
> hi everyone,
> this is going to be a lengthy email so i'd like to thank those
> who will read it and maybe give me their opinions.
> The topic here is throwing some ideas for improving QtQuick event
> handling (both for mouse and keyboard).
> First let's take a look at some simple/silly keyboard examples that will
> make us introduce the main topic and some issue with the current way of handling
> events.
> First example (here)[http://pastebin.com/jiGJimGS]. Here the user would like to 
> count how many times the right key has been pressed. Try it and you will see that
> the Keys.onPressed handler will be called twice when the caret is at the end
> of the TextField but it will work when the caret moves inside the text. I know
> that once the final user discover this, he can workaround it and fix the counter but 
> this is not the point. The reason why the Keys.onPressed is called twice 
> is caused by the TextField implementation. In fact a TextField is a composed Control
> made of a TextInput plus a Text. At the same time, the end user doesn't know this and

We have mostly stopped working on QtQuick Controls 1, because the implementation of Controls 2 is much more performant, and we have to focus on that.  It was achieved by implementing behavior in C++, so each Control is composed of fewer objects.  You should be testing it for some time already.  It’s already out of tech preview, fully supported in 5.7.  (Unfortunately the tech preview period was pretty short, from 5.6.0 to 5.7.0.)  So please see what kinds of issues you still have with that.

> he can only interact with the whole TextField (in theory he doesn't have to know that
> the TextField contains a inner TextInput). For this reason the developer of the TextField
> control added a Keys.forward[root] inside the inner TextInput toward the parent TextField.
> This is correct in the current state of things because this allows the end user to override
> the key handling of the the TextField. But why we see two events? This comes by the fact that:
> 1 The right pressed event is delivered to the inner TextInput (because it has activeFocus)
> 2 The event is forward to the TextField before being processed by the TextInput (because of Keys.forward[])
> 3 The TextField calls the Keys.onPressed handler of the user (that simply ignores the event)
> 4 The event is processed by the TextInput that ignores it (because the caret is at the end)
> 5 The event bubble up to the TextField again and the Keys.onPressed event handler will be called for the second time.
> This example shows us that the actual way of handling events doesn't play well with item composition.
> Furthermore the event forwarding is very error prone given that without attention is very easy to create
> loops or unexpected chain of calls (due to bubbling and forwarding).
> Obviously there could be several ways to solve this, for example
> 1) Find some way of disabling the call to the event handler (during bubbling) since it has been called once
> 2) Find some way for making a complex object (made of sub controls) to be seen as a unique entity for the sake
> of event handling
> 3) Let the user monitor the events before they reach a child.
> 4) <--- your ideas here
> Let's take a look to another example (here)[http://pastebin.com/9s36E6bm]. Here the user would like to disable 
> the ComboBox event handling, for example because the ComboBox is inside a ListView or TableView and he doesn't
> want to break the View event handling (up ad down keys). If you try it you will see that it doesn't work. Even if
> the events are accepted they're still delivered to the ComboBox. This is due (probably) by the fact the a Keys.forward[]
> directive is missing inside the ComboBox implementation. 
> Another broader example is allowing a ListView to handle mouse clicks and events even if the delegate has a MouseArea on top.
> This letter case can be implemented by using the filterChildMouseEvents api.
> I've taken a look at what other frameworks do, in particular WPF (this doesn't mean the design of WPF is better or clever).
> In WPF it seems there are three mechanism 
> 1) Event tunneling. The events first make a down path from the UI root node to the target node
> 2) Event bubbling. The events are bubbled from the target node up to the UI root node
> 3) Event flagged as handled (accepted) still make their way up (bubbling) because some nodes could be interested in a event 
> even if it has been flagged (rare ma usefull).
> Point 1 it's basically a child monitor. Point 2 is what we already do. Point 3 is usefull because at point 1 you know that
> an event is being delivered to a child but you don't know if it'll be accepted or not. Point 3 can be used for this purpose.

OK, that’s an interesting idea.  I was thinking that there must be some good ideas from other frameworks, but am not sure which one has best-in-class handling of events, especially mouse and/or touch (more on that below), and don’t have time to learn them all.

> Just for fun i pushed two wip code reviews for event tunneling for keys events: one in QtCore https://codereview.qt-project.org/#/c/163502/
> and one in QtDeclarative https://codereview.qt-project.org/#/c/163502/ 
> With this patch the example 1 and 2 can be written like this: http://pastebin.com/t0ewNrxC and http://pastebin.com/LqvPZkZN
> Further informations about tunneling and wpf event handling can be found here https://msdn.microsoft.com/it-it/library/ms742806(v=vs.110).aspx#how_event_processing_works
> My patches are just wips and if there's interest i can work further on them. For example i would like
> to have some sort of Mouse attacched object, like Keys and implementing bubbling even for handled events.
> Finally maybe we need some sort of QEP for writing long term ideas and discuss them in public.

We are working on a new QPointerEvent and a set of pointer handler objects now.  My main goal is to unify the handling of mouse, touch and tablet events, and make it possible for a handler either to be device-agnostic (just handle a click or tap, don’t distinguish them) or to filter events based on devices, capabilities etc. (handle a touch-drag differently than a stylus-drag).  Those filters are specified declaratively.  The design for the handlers is between that of MouseArea and the Keys handler: it’s not an attached object, because I think you need to be able to have multiple handlers per Item; but it’s just an object, not an Item.  It is a delegate object for handling a particular kind of event within an Item.  And in the context of a single Item, all handlers have equal opportunity to handle every event.  This amounts to a loosening of the restriction that an Item must grab an event in order to get the updates: at least it’s only the Item grabbing, but multiple handlers inside can still get the updates.  So you can easily use a PinchHandler and a DragHandler and a TapHandler together, as siblings declared within one Item.  But I think we might still need to go further than that, because sometimes it’s unavoidable that you need events to bubble up through a stack of Items, and so far to do that you set filtersChildMouseEvents.  And child-filtering feels a bit too special, too out-of-band.  Or, you can write a QObject event filter, which also seems like dubious design to me, but the jury’s still out on whether we ought to consider that technique a fundamental part of event handling in Qt Quick.

We have discussed sometimes whether declared Handlers ought to be augmented with attached objects, so that handling the mouse or touchpoint could be a one-liner in some cases.  But attached objects are actually less lightweight in implementation (I wish that wasn’t the case, but it always has been); and if you can only attach one handler of a particular type to an Item, then some advanced cases would require imperative code like what you’d typically write in Keys.onPressed.  That kind of code is not declarative enough, IMO.  It’s mitigated somewhat by all the key-specific signals like onSpacePressed, onTabPressed etc.  But we don’t yet have signals for every possible key; and even if you can avoid writing a big switch in Keys.onPressed, you might still need to check the modifiers inside your onSpacePressed (or whatever) function.

We also came up with the idea of handling pointer events in phases: for each Handler which is visited, first ask if it wants the event, then ask it to process the event, then ask it to post-process the event (this last step is an opportunity to emit all signals at the same time and in the right order).  But so far we are going through those phases while visiting one handler - not visiting all of them to ask whether it wants it, and then visiting all of them again to process it.  I had the idea because I don’t like the ambiguity that QQuickItem::event() returns a bool, and there’s also the opportunity to accept or reject the event: it’s not sufficiently memorable which of those is the right way to prevent or allow propagation in a particular situation, and which takes precedence.  Every time I look at the code I have to figure that out again.  Whereas if the prefilter returns false, at the very least it means that it’s going to propagate this time, and maybe it should also mean that this handler does not see a need for a grab… or maybe not.  We have the opportunity to revisit the grabbing concept now.

I agree that we should try to bring key and pointer handling closer.  But I’m also less aware of what’s wrong with key handling, compared to pointer handling.  So it’s worthwhile to discuss, but my main focus right now is pointer handling, to get it done in time for a tech preview for 5.8 (that means there’s only a few weeks left before feature freeze).  Later we could try to apply what we learn to key handling, to the extent that it’s applicable.

Everyone is welcome to send me challenges: mouse and touch use cases that are tough to implement with the current Areas.  I know we have quite a few bugs about those cases already.  I’m not sure if we can come up with declarative syntax that makes every such case easy to express, but it’s a stretch goal.

More information about the Development mailing list