[Development] Input events handling ideas and feedback

[Filippo Cucchetto]

Thank you Shawn for your reply

> Control is composed of fewer objects. You should be testing it for some
time already.
Well i already used the QtQuickControls 2 and i do understand that their
design if superior (like for example the font handling) but
i still think that composition of multiple QQuickItems is something we
should deal with (independently by the fact that we
support Native or not Native styles).

> don’t have time to learn them all
I'm sorry if i sounded rude, i didn't mean that you (or who worked in the
QtQuick event handling implementation) should know
all frameworks implementations. I just wanted to give some hints, discuss
them together and show some WIP patches for implementing
something similar. Obviously i'm totaly open to drop them if not
appreciated or against the future visions/plans for QtQuick. The more
we discuss these things openly the more we can arrive at a better result.

> My main goal is to unify the handling of mouse, touch and tablet events..
> ...even if you can avoid writing a big switch in Keys.onPressed,
I agree with that even if IMHO having a javascript switch in the handler
it's not so terrible. To me what matter is to make most use cases
possible in a "good" way.

>The design for the handlers is between that of MouseArea and the Keys
handler: it’s not an attached object, because I think you need to be able
to have multiple handlers per Item; but it’s just an object, not an Item.
> It is a delegate object for handling a particular kind of event within an
Item.  And in the context of a single Item, all handlers have equal
opportunity to handle every event.
> This amounts to a loosening of the restriction that an Item must grab an
event in order to get the updates: at least it’s only the Item grabbing,
but multiple handlers inside can still get the updates.
This seems reasonable but still (as you exaplained) we have the problem
with bubbling and the need for monitoring mouse/keys event in a parent item.
To me it seems that tunneling/bubbling methods are not in contrast with
your idea. Tunneling/bubbling handle the recursive phase; instead multiple
handlers (as you suggested) handle the item step.

> Everyone is welcome to send me challenges: mouse and touch use cases that
are tough to implement with the current Areas.
The most common hard thing to implement right now is handling
ListView/TableView/TreeViews mouse events. The problem is that an item
delegate that handles mouse events break the view mouse handling.
Obviously rejecting the MouseEvent in the pressEvent is not sufficient: in
fact this allows the event to bubble up to the view but prevent the
delegate MouseArea to receive composed events like clicks or double clicks.
To me a possible solution would be to use point (3) of my previous mail:
the View should receive all the mouse events independently by the fact that
they have been handled or not. In this way the view can handle
selection/dragging correctly. Obviously this behaviour could be unwanted,
but this could be disable with something like this
ListView {
    Mouse.onPressed: {
        if (mouse.handled) {
            return
        }
    }
}

The idea is to have a possible Mouse attached object that has a higher
priority over the normal item event handling (like is done with the Keys
attached objects). In this
way we can disable the item default event handling with already handled
events (that bubbled up from a delegate).

Thank you again for the answer

F.

2016-06-27 10:39 GMT+02:00 Shawn Rutledge <Shawn.Rutledge at qt.io>:

>
> > On 25 Jun 2016, at 15:15, Filippo Cucchetto <filippocucchetto at gmail.com>
> wrote:
> >
> > hi everyone,
> > this is going to be a lengthy email so i'd like to thank those
> > who will read it and maybe give me their opinions.
> > The topic here is throwing some ideas for improving QtQuick event
> > handling (both for mouse and keyboard).
> > First let's take a look at some simple/silly keyboard examples that will
> > make us introduce the main topic and some issue with the current way of
> handling
> > events.
> > First example (here)[http://pastebin.com/jiGJimGS]. Here the user would
> like to
> > count how many times the right key has been pressed. Try it and you will
> see that
> > the Keys.onPressed handler will be called twice when the caret is at the
> end
> > of the TextField but it will work when the caret moves inside the text.
> I know
> > that once the final user discover this, he can workaround it and fix the
> counter but
> > this is not the point. The reason why the Keys.onPressed is called twice
> > is caused by the TextField implementation. In fact a TextField is a
> composed Control
> > made of a TextInput plus a Text. At the same time, the end user doesn't
> know this and
>
> We have mostly stopped working on QtQuick Controls 1, because the
> implementation of Controls 2 is much more performant, and we have to focus
> on that.  It was achieved by implementing behavior in C++, so each Control
> is composed of fewer objects.  You should be testing it for some time
> already.  It’s already out of tech preview, fully supported in 5.7.
> (Unfortunately the tech preview period was pretty short, from 5.6.0 to
> 5.7.0.)  So please see what kinds of issues you still have with that.
>
> > he can only interact with the whole TextField (in theory he doesn't have
> to know that
> > the TextField contains a inner TextInput). For this reason the developer
> of the TextField
> > control added a Keys.forward[root] inside the inner TextInput toward the
> parent TextField.
> > This is correct in the current state of things because this allows the
> end user to override
> > the key handling of the the TextField. But why we see two events? This
> comes by the fact that:
> > 1 The right pressed event is delivered to the inner TextInput (because
> it has activeFocus)
> > 2 The event is forward to the TextField before being processed by the
> TextInput (because of Keys.forward[])
> > 3 The TextField calls the Keys.onPressed handler of the user (that
> simply ignores the event)
> > 4 The event is processed by the TextInput that ignores it (because the
> caret is at the end)
> > 5 The event bubble up to the TextField again and the Keys.onPressed
> event handler will be called for the second time.
> > This example shows us that the actual way of handling events doesn't
> play well with item composition.
> > Furthermore the event forwarding is very error prone given that without
> attention is very easy to create
> > loops or unexpected chain of calls (due to bubbling and forwarding).
> > Obviously there could be several ways to solve this, for example
> > 1) Find some way of disabling the call to the event handler (during
> bubbling) since it has been called once
> > 2) Find some way for making a complex object (made of sub controls) to
> be seen as a unique entity for the sake
> > of event handling
> > 3) Let the user monitor the events before they reach a child.
> > 4) <--- your ideas here
> > Let's take a look to another example (here)[http://pastebin.com/9s36E6bm].
> Here the user would like to disable
> > the ComboBox event handling, for example because the ComboBox is inside
> a ListView or TableView and he doesn't
> > want to break the View event handling (up ad down keys). If you try it
> you will see that it doesn't work. Even if
> > the events are accepted they're still delivered to the ComboBox. This is
> due (probably) by the fact the a Keys.forward[]
> > directive is missing inside the ComboBox implementation.
> > Another broader example is allowing a ListView to handle mouse clicks
> and events even if the delegate has a MouseArea on top.
> > This letter case can be implemented by using the filterChildMouseEvents
> api.
> > I've taken a look at what other frameworks do, in particular WPF (this
> doesn't mean the design of WPF is better or clever).
> > In WPF it seems there are three mechanism
> > 1) Event tunneling. The events first make a down path from the UI root
> node to the target node
> > 2) Event bubbling. The events are bubbled from the target node up to the
> UI root node
> > 3) Event flagged as handled (accepted) still make their way up
> (bubbling) because some nodes could be interested in a event
> > even if it has been flagged (rare ma usefull).
> > Point 1 it's basically a child monitor. Point 2 is what we already do.
> Point 3 is usefull because at point 1 you know that
> > an event is being delivered to a child but you don't know if it'll be
> accepted or not. Point 3 can be used for this purpose.
>
> OK, that’s an interesting idea.  I was thinking that there must be some
> good ideas from other frameworks, but am not sure which one has
> best-in-class handling of events, especially mouse and/or touch (more on
> that below), and don’t have time to learn them all.
>
> > Just for fun i pushed two wip code reviews for event tunneling for keys
> events: one in QtCore https://codereview.qt-project.org/#/c/163502/
> > and one in QtDeclarative https://codereview.qt-project.org/#/c/163502/
> > With this patch the example 1 and 2 can be written like this:
> http://pastebin.com/t0ewNrxC and http://pastebin.com/LqvPZkZN
> > Further informations about tunneling and wpf event handling can be found
> here
> https://msdn.microsoft.com/it-it/library/ms742806(v=vs.110).aspx#how_event_processing_works
> > My patches are just wips and if there's interest i can work further on
> them. For example i would like
> > to have some sort of Mouse attacched object, like Keys and implementing
> bubbling even for handled events.
> > Finally maybe we need some sort of QEP for writing long term ideas and
> discuss them in public.
>
> We are working on a new QPointerEvent and a set of pointer handler objects
> now.  My main goal is to unify the handling of mouse, touch and tablet
> events, and make it possible for a handler either to be device-agnostic
> (just handle a click or tap, don’t distinguish them) or to filter events
> based on devices, capabilities etc. (handle a touch-drag differently than a
> stylus-drag).  Those filters are specified declaratively.  The design for
> the handlers is between that of MouseArea and the Keys handler: it’s not an
> attached object, because I think you need to be able to have multiple
> handlers per Item; but it’s just an object, not an Item.  It is a delegate
> object for handling a particular kind of event within an Item.  And in the
> context of a single Item, all handlers have equal opportunity to handle
> every event.  This amounts to a loosening of the restriction that an Item
> must grab an event in order to get the updates: at least it’s only the Item
> grabbing, but multiple handlers inside can still get the updates.  So you
> can easily use a PinchHandler and a DragHandler and a TapHandler together,
> as siblings declared within one Item.  But I think we might still need to
> go further than that, because sometimes it’s unavoidable that you need
> events to bubble up through a stack of Items, and so far to do that you set
> filtersChildMouseEvents.  And child-filtering feels a bit too special, too
> out-of-band.  Or, you can write a QObject event filter, which also seems
> like dubious design to me, but the jury’s still out on whether we ought to
> consider that technique a fundamental part of event handling in Qt Quick.
>
> We have discussed sometimes whether declared Handlers ought to be
> augmented with attached objects, so that handling the mouse or touchpoint
> could be a one-liner in some cases.  But attached objects are actually less
> lightweight in implementation (I wish that wasn’t the case, but it always
> has been); and if you can only attach one handler of a particular type to
> an Item, then some advanced cases would require imperative code like what
> you’d typically write in Keys.onPressed.  That kind of code is not
> declarative enough, IMO.  It’s mitigated somewhat by all the key-specific
> signals like onSpacePressed, onTabPressed etc.  But we don’t yet have
> signals for every possible key; and even if you can avoid writing a big
> switch in Keys.onPressed, you might still need to check the modifiers
> inside your onSpacePressed (or whatever) function.
>
> We also came up with the idea of handling pointer events in phases: for
> each Handler which is visited, first ask if it wants the event, then ask it
> to process the event, then ask it to post-process the event (this last step
> is an opportunity to emit all signals at the same time and in the right
> order).  But so far we are going through those phases while visiting one
> handler - not visiting all of them to ask whether it wants it, and then
> visiting all of them again to process it.  I had the idea because I don’t
> like the ambiguity that QQuickItem::event() returns a bool, and there’s
> also the opportunity to accept or reject the event: it’s not sufficiently
> memorable which of those is the right way to prevent or allow propagation
> in a particular situation, and which takes precedence.  Every time I look
> at the code I have to figure that out again.  Whereas if the prefilter
> returns false, at the very least it means that it’s going to propagate this
> time, and maybe it should also mean that this handler does not see a need
> for a grab… or maybe not.  We have the opportunity to revisit the grabbing
> concept now.
>
> I agree that we should try to bring key and pointer handling closer.  But
> I’m also less aware of what’s wrong with key handling, compared to pointer
> handling.  So it’s worthwhile to discuss, but my main focus right now is
> pointer handling, to get it done in time for a tech preview for 5.8 (that
> means there’s only a few weeks left before feature freeze).  Later we could
> try to apply what we learn to key handling, to the extent that it’s
> applicable.
>
> Everyone is welcome to send me challenges: mouse and touch use cases that
> are tough to implement with the current Areas.  I know we have quite a few
> bugs about those cases already.  I’m not sure if we can come up with
> declarative syntax that makes every such case easy to express, but it’s a
> stretch goal.
>
>


-- 
Filippo Cucchetto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20160627/6033f7b9/attachment.html>