[Development] Input events handling ideas and feedback

[Shawn Rutledge]

> On 27 Jun 2016, at 23:12, Filippo Cucchetto <filippocucchetto at gmail.com> wrote:
> 
> Thank you Shawn for your reply
> 
> > Control is composed of fewer objects. You should be testing it for some time already.
> Well i already used the QtQuickControls 2 and i do understand that their design if superior (like for example the font handling) but
> i still think that composition of multiple QQuickItems is something we should deal with (independently by the fact that we
> support Native or not Native styles).

Yes that’s true.  There are lots of third-party sets of controls, and there’s nothing wrong with creating them by composition - it’s just that QQControls 1 got a bit carried away with creating too many objects.

> > don’t have time to learn them all
> I'm sorry if i sounded rude, i didn't mean that you (or who worked in the QtQuick event handling implementation) should know
> all frameworks implementations.

You were not at all rude, and I didn’t mean to be either.  When it comes to event propagation/grabbing/stealing/monitoring, our way is just one of many possibilities, but maybe it could be more elegant.  So you provided one example (WPF), and I’ll try to think more about the pros and cons of how they did it.  If we had time to filter out all the good ideas from all the frameworks, we’d have a better chance of making ours the best.  ;-)

> I just wanted to give some hints, discuss them together and show some WIP patches for implementing
> something similar. Obviously i'm totaly open to drop them if not appreciated or against the future visions/plans for QtQuick. The more
> we discuss these things openly the more we can arrive at a better result.

Yes.

> > My main goal is to unify the handling of mouse, touch and tablet events..
> > ...even if you can avoid writing a big switch in Keys.onPressed,
> I agree with that even if IMHO having a javascript switch in the handler it's not so terrible. To me what matter is to make most use cases
> possible in a "good" way.

Well we advertise that it’s a declarative language.  We could make it even more that way, even if it’s not strictly necessary.  Would it be more elegant?

> >The design for the handlers is between that of MouseArea and the Keys handler: it’s not an attached object, because I think you need to be able to have multiple handlers per Item; but it’s just an object, not an Item.  
> > It is a delegate object for handling a particular kind of event within an Item.  And in the context of a single Item, all handlers have equal opportunity to handle every event.  
> > This amounts to a loosening of the restriction that an Item must grab an event in order to get the updates: at least it’s only the Item grabbing, but multiple handlers inside can still get the updates.
> This seems reasonable but still (as you exaplained) we have the problem with bubbling and the need for monitoring mouse/keys event in a parent item.
> To me it seems that tunneling/bubbling methods are not in contrast with your idea. Tunneling/bubbling handle the recursive phase; instead multiple handlers (as you suggested) handle the item step.
> 
> > Everyone is welcome to send me challenges: mouse and touch use cases that are tough to implement with the current Areas.
> The most common hard thing to implement right now is handling ListView/TableView/TreeViews mouse events. The problem is that an item delegate that handles mouse events break the view mouse handling.

> Obviously rejecting the MouseEvent in the pressEvent is not sufficient: in fact this allows the event to bubble up to the view but prevent the delegate MouseArea to receive composed events like clicks or double clicks.

Well in my experience, handling clicks inside delegates works fine, but we have trouble with hover propagation, and we have trouble with draggable items inside the delegate, especially when you try to drag via touch.

The ListView/etc. monitors its children’s events.  So if you drag beyond the drag threshold in the ListView’s direction of interest, the ListView steals the grab, and for the MouseArea inside the delegate, the grab is canceled.

I think we could say that QQuickWindowPrivate::deliverInitialMousePressEvent() does tunnelling: it starts with the root and goes recursively _down_ through the children until the event is accepted (Handled in WPF terminology).  But for each child recursively, it calls QQuickWindow::sendEvent(), which in the case of mouse events calls sendFilteredMouseEvent(item->parentItem(), item, e, &hasFiltered).  And that goes recursively _up_ the parent hierarchy, calling childMouseEventFilter on any ancestor Item which has the filtersChildMouseEvents flag set.  So it’s kindof like bubbling, but it happens before the real event handling in the leaf Item, rather than afterwards. And it happens multiple times per “target", which would be inefficient and redundant, so QSet<QQuickItem *> *hasFiltered is used to keep track of the items that have already had their chance at filtering.  (Yeah, a fresh new QSet gets populated for every event.  But it’s not the only case of that, either.)  So that’s already rather like bubbling, but I'd worry about changing the order: WPF gives the “source” a chance to handle the event first, then the bubbling happens after; whereas QtQuick does sendFilteredMouseEvent first and then lets the Item have the event.  That might be a big change to make.  But then again, by the time the ListView’s drag threshold is exceeded, the MouseArea inside the delegate has already been the grabber for a while; so the grab must be canceled, and that involves sending another event to tell it that it was canceled.  So if we did tunnelling followed by bubbling, we’d need to keep doing that, and maybe it would be OK.

https://msdn.microsoft.com/en-us/library/ms742806(v=vs.110).aspx#how_event_processing_works is the English equivalent to the link you posted.  So now I have some quotes and comments.

They start with an example with 3 buttons inside a couple of nested containers, and say "the source of a Click event is one of the Button elements”.  Saying that the event source is the leaf node, or that the leaf node “raised” the event, hides how it figured that out.  Didn’t it need to use tunnelling to decide that?  At least from Qt’s perspective, the Window “raised” the event, and we need to find the relevant Item inside.  But they say tunnelling is one routing strategy alongside “bubbling” and “direct”, and routed events use “one of three routing strategies”.  

Well, maybe they used tunnelling, or some other means to narrow down the items in the neighborhood where the event occurred, to find the “source”, before this routing even starts.  So the “source” is always a leaf? 

I was thinking of trying some kind of space partitioning; didn’t get around to trying that.  Items which have Handlers could be put into a k-d tree, or something.  But there would have to be a worthwhile speedup to justify maintaining that structure.  And we’d only need it for press events, not for releases and updates, as long as we keep the grabbing the same.

Then under WPF Input Events there is a tree of elements.  Based on the explanation, I guess it does start with the root element and does tunnelling, but is that the first step or do they already know by magic that leaf #2 “raised” the event?  How did they skip the step of tunnelling to leaf #1?

"Usually, once the input event is marked Handled, further handlers are not invoked. … The exception to this general statement about Handled state is that input event handlers that are registered to deliberately ignore Handled state of the event data would still be invoked along either route.”  

Aha… so bubbling doesn’t always occur: the item must register an interest in receiving already-handled events, right?  I was thinking of having a list of Items like that in QQuickWindow: those which want to monitor all pointer events regardless of the grabber.  Or, maybe making it so that grabbing can be non-exclusive: multiple items can grab at the same time, but it’s prioritized so that the leaf item will get the event first, or the “grabbiest” will get it first, whatever that means.

So there is a registry for items which want to monitor all events, but the bubbling should still be done in reverse-z-order?  After the tunnelling, the leaf gets the event, then we look at its parent and see if it’s in the monitor-all list, and let it have the bubbled event if so; then go to its parent and do the same; etc.  Does that sound about right?  Well, instead of a separate list, we have the filtersChildMouseEvents flag, and we are checking each parent.

> To me a possible solution would be to use point (3) of my previous mail: the View should receive all the mouse events independently by the fact that they have been handled or not. In this way the view can handle
> selection/dragging correctly. Obviously this behaviour could be unwanted, but this could be disable with something like this
> ListView {
>     Mouse.onPressed: {
>         if (mouse.handled) {
>             return
>         }
>     }
> }

Returning from a JS function will happen whether you write “return” or not.

There’s an escalation/arms-race problem: by default the leaf MouseArea grabs, which should already mean that it’s exclusive, it doesn’t want any ancestor to see the event.  (Why should grabbing be default?  Only because subsequent events can be delivered quickly, it seems.  But grabbing prevents easy monitoring by the ancestors.)  But, we violate that by having the ListView monitor its children’s events - not only that, it gets them first, before the children.  And it can also steal the grab.  Oh, but the child can also have preventStealing: true.  We don’t yet have ListView.preventStealingPrevention or stealEvenMoreBrazenly.  ;-)

So should the leaf item not grab by default?  Well then we need to do more work to deliver subsequent events: redo the “tunnelling” step each time, right?

Anyway I guess we have bubbling, within the parent hierarchy, in the form of setting filtersChildMouseEvents to true.  What we don’t have is event propagation to other Items which occupy the same space but are siblings or cousins of that hierarchy.

Whereas with key events, deliverKeyEvent sends to activeFocusItem only.  It’s the keyboard grabber, and that had to be determined ahead of time.  That’s different than mouse and touch handling.  But I guess it’s because only one TextInput is supposed to have an active cursor, so you can see where it’s going to go.  The delivery is not based on position and geometry.  You just run into trouble when you use key events for other purposes than text input, right?

> The idea is to have a possible Mouse attached object that has a higher priority over the normal item event handling (like is done with the Keys attached objects). In this
> way we can disable the item default event handling with already handled events (that bubbled up from a delegate).

So pre-filtering like we already do when filtersChildMouseEvents is true, but outside the hierarchy, so that siblings can monitor mouse events too?  Only via the attached object?

For high-priority key handling, we have Shortcut.  Those are handled early in  QGuiApplicationPrivate::processKeyEvent() before the QKeyEvent is constructed and delivered.

And there is the QObject::installEventFilter() technique.

We could have a PointerFilter if we like, or certain specialized handlers for specific use cases.  It could install itself on whatever object it’s declared inside of, or maybe it has to install itself on the window.  But then there wouldn’t necessarily be a way to control the ordering if you declare multiple filters, and they are all on the Window.