[Interest] Astropad and Tablet events

[Alexei Gilchrist]

On 22 Mar 2016, at 20:18, Rutledge Shawn wrote:
>> On 22 Mar 2016, at 02:30, Alexei Gilchrist <te100 at runbox.com> wrote:
>>
>> Hi all,
>>
>> I'm trying to use Astropad (app provides OsX tablet interface with an 
>> iPad) with an app I’ve written and I'm getting odd results with 
>> tablet events in QT. First of all, I've been in contact with the devs 
>> and they seem to think it's a QT bug rather than an Astropad bug, and 
>> in any case Astropad works with other graphics programs that I've 
>> tried.
>
> Why do they think it’s a Qt bug?  Maybe they should tell us what we 
> are doing wrong if they know.
>
> I guess there must be a Mac daemon which tries to emulate a Wacom by 
> sending the same events?  It’s not only for Qt applications, right?  
> But Qt applications work with a real Wacom but not Astropad, so they 
> must not be emulating the events exactly the same.

It is running a daemon that handles communication with the corresponding 
app on an iPad. One reason to believe it’s a QT bug is that Astropad 
works fine with other applications such as Gimp, SketchbookEpress, 
InKist, Photoshop, all respond to pressure variations so multiple 
toolkits are able to interpret the synthesised events correctly.

> You can enable the logging category qt.qpa.input.tablet to see how the 
> events come in (the code is in qnsview.mm).

Cool. Didn’t know that. I Need to figure out how to display the 
logging events in pyqt as setting

export QT_LOGGING_RULES='qt.qpa.input=true'

or editing ~/.config/QtProject/qtlogging.ini is not reporting to the 
console, maybe the python logging module which I’m using is taking 
over stderr. Will play around and see if I can get some feedback.

>
>> I'm coding in python using pyqt and have Python 3.5.0 and QT 5.5.0 
>> installed.
>>
>> At the level of QWidget you get an initial TabletPress event with 
>> pressure=0 followed immediately by a TabletRelease event with 
>> pressure>0 at which point the rest of the stroke is only reported as 
>> mouse events, with are low frequency and don’t contain pressure 
>> info.
>>
>> Astropad:
>>
>> TabletPress: pointer=1 pressure=0.0 tilt=(27,12)
>> TabletRelease: pointer=1 pressure=0.13333334028720856 tilt=(27,12)
>> MousePress
>> MouseMove
>> MouseMove
>> ...[ many MouseMove ]...
>> MouseRelease
>>
>> Works as you'd expect with Wacom tablet, though accepting the tablet 
>> events does't stop the mouse events being fired (minor niggle):
>
> That’s https://bugreports.qt.io/browse/QTBUG-47007  It’s probably 
> always been that way, at least in Qt 5, because the platform plugins 
> are doing this mouse emulation when necessary.  (In some cases the 
> window system will send a synthesized mouse event itself, in which 
> case we don’t need to.)  Plugins don’t pay attention to whether 
> the tablet event was accepted or not, because sending events from the 
> platform plugin into the application is asynchronous, going through a 
> queue.  We assume the tablet event is rejected because that’s the 
> default policy (outside of QtQuick where we assume an event is 
> accepted), so the default is that the application does nothing, and 
> then it gets a mouse event (so that you can use the stylus to interact 
> with widgets etc.)  AFAIK there isn’t a nice mechanism to wait and 
> see whether the app accepted the tablet event or not; we’d have to 
> wait until the asynchronous delivery is done, somehow.
>
> In older Qt versions, you needed the mouse event (which means you 
> should reject the tablet event, according to the docs) in order to 
> find out what buttons on the stylus were pressed.  But if you reject 
> the tablet press event, you don’t get movement updates as tablet 
> events, and the mouse events are compressed (as you noticed).  Now, 
> QTabletEvent has buttons too, so you no longer need to do that, but 
> there may still be some legacy applications that are already written 
> to work that way.  So I’m not sure whether we should fix that now, 
> or will it be seen as a behavior change which breaks applications?  I 
> guess fixing it in 5.7 or later ought to be OK at least.  But it’s 
> nontrivial to fix.

This is not much of a problem as a TabletPress is delivered before a 
MousePress so it’s easy to toggle a pen_down variable and ignore mouse 
events.

>
> QTBUG-51617 is about labelling the mouse events as synthesized so that 
> you can at least easily know which mouse events to reject.  It should 
> be fixed now on X11 (for 5.6.1), we just need to make sure the other 
> platforms work consistently.  So you can have a mouse handler which 
> ignores such events.

Will take a look at those, it would be a more robust solution in case 
the ordering of events should change.

>
>> TabletPress: pointer=1 pressure=0.4313725531101227 tilt=(0,0)
>> MousePress
>> TabletMove: pointer=1 pressure=0.45098039507865906 tilt=(0,0)
>> MouseMove
>> TabletMove: pointer=1 pressure=0.45098039507865906 tilt=(0,0)
>> TabletMove: pointer=1 pressure=0.4470588266849518 tilt=(0,0)
>> ...[ many MouseMove and many more TableMove ] ...
>> TabletRelease: pointer=1 pressure=0.0 tilt=(0,0)
>> MouseMove
>> MouseRelease
>>
>> At the level of QApplication I'm seeing TabletEnterProximity and 
>> TabletLeaveProximity events as you'd expect.
>
> Cool.  Are you using an iPad Pro?  Otherwise it can’t detect 
> proximity, but maybe it synthesizes those events when you press?

Yes, using an iPad pro. Astropad mirrors the mac screen to the iPad pro 
and captures pencil and finger events synthesising corresponding events 
on the mac. Effectively creating a Cintiq like tablet. I’m guessing 
they add the proximity events for compatibility with apps that are coded 
for a wacom tablet and looking for the hover (iPad pro does’t detect 
pencil proximity).

When I first contacted the Astropad devs you could trick their code into 
sending some sequence that QT transmitted with TabletMove events for one 
stroke after doing a stroke on a wacom tablet. I pointed out the problem 
and sent then a test app and got back:

> I wanted to let you know that we found a workaround for this. It does 
> appear to be a bug in QT, but we figured out what event from Astropad 
> was confusing it. By removing this event (which doesn’t appear to be 
> necessary) QT once again works with Astropad.

Since then, they’ve updated their code and no TabletMove events are 
generated at all. Not sure why they concluded that they’d created a 
workaround, but they’re no longer responding to me pestering them :P

Alexei