[Development] Multitouch on top of XInput 2.2

[Shawn Rutledge]

On Monday wk28 2012 July 09 07:42:02 PM Shawn Rutledge wrote:
> On Thursday wk26 2012 June 28 05:09:41 PM ext Shawn Rutledge wrote:
> > On 28 June 2012 13:03, Christian Spielberger
> > 
> > <c.spielberger at bct-electronic.com> wrote:
> > > Hi,
> > > 
> > > I got multitouch on top of XInput 2.2 to work.
> > > 
> > > See http://www.youtube.com/watch?v=sRSC4CEYmIg !
> > > 
> > > Repo:
> > > See https://gitorious.org/~cspiel/qt/cspiels-qt-4_8-with-xinput2_1 !
> > > 
> > > Branch:
> > > 4.8-XI2.2-touch
> > > 
> > > Maybe the branch can be the base for the multitouch integration for X11.
> > 
> > Excellent!  I was planning to work on Qt 5 support for XI 2.2 soon, so
> > that should be helpful.
> 
> However due to the way the code has been rearranged, the patch will be in
> different files anyway.  

To be more specific: due to the introduction of QPA it's quite different; in 
Qt 5 the platform XCB plugin generates touchpoints of type struct TouchPoint 
to put into a QWindowSystemInterfacePrivate::TouchEvent, whereas in your patch 
you can generate a QTouchEvent::TouchPoint directly.  However from reading 
your code I learned of the existence of the pre-converted coordinates:

touchPoint.setPos( QPointF(xideviceevent->event_x, xideviceevent->event_y) );
touchPoint.setScreenPos( QPointF(xideviceevent->root_x, xideviceevent->root_y) 
);

as opposed to having to get them from the valuators and convert them manually.  
But there are some odd things about it: you used XLib so you get them as 
doubles, but in xcb they have 16.16 fixed-point format, so I have convert them 
to qreals.  

Another oddity is that when testing a touchscreen, each xXIDeviceEvent from 
xcb has the correct coordinates already, whereas with an Apple Trackpad, both 
of them will be at the same point where the cursor is.  Maybe that's because 
the touchpad is a relative device rather than absolute, or maybe it's an xcb 
bug?  Because what I think I want is that the first touchpoint should be at 
the cursor position, and the second one should be offset the same distance 
from the first as my second finger is offset on the trackpad.  But with the 
current version of my patch, if I run the fingerpaint example, all the fingers 
are superimposed, so you just get different ellipses in different colors 
painted on top of each other.  (At least the major/minor axes are unique 
depending on pressure.)  So I'm thinking of calculating the offset from the 
converted raw point to the given root_x and root_y for the first point, and 
then somehow offsetting the second and subsequent points.  But I'm not sure if 
it's right for Qt to do that; if it's a bug or oversight in xcb (or somewhere 
else), maybe it will get fixed there eventually.

> Would you like to convert your changes into a patch
> on gerrit so it can be reviewed to go into 4.8.x?  I can also test it on a
> couple of devices.

I noticed you deleted the code to deal with the valuators completely, so that 
means in your branch there is no support for pressure or major/minor axes, 
right?  When I run fingerpaint against your branch, it is indeed insensitive 
to pressure, I just get circles all the same size.  I don't think you should 
be ignoring all the valuators, even though you are correctly making use of the 
given points from the event itself.

The axes are supposed to describe the size and shape of the elliptical contact 
patch of your finger against the touch device.  I have tested my Qt 5 patch on 
the following devices:

Lenovo S10-3t: has neither pressure nor axes; 2 points max

Apple Bluetooth trackpad: has major and minor axes plus (bogus?) orientation, 
5 points max; points from the XI2 event are all at the same position as the 
cursor 

Apple trackpad on a Macbook Pro (17" from 2009 or so): has major and minor 
axes plus ABS Pressure (NOT Abs MT Pressure), 5 points max; forgot whether it 
has orientation; points from the XI2 event are all at the same position as the 
cursor 

ExoPC: no axes, has Abs MT Pressure but sometimes I have seen it go to the 
minimum possible value if there are 2 fingers.  10 points max.

What I mean by "bogus orientation" on the trackpad is that instead of having 
per-finger orientation, it exists only if 2 or more fingers are pressed, so it 
would seem to be related to the angle between them; but it's quite jumpy, and 
I'm not sure what a range from -31 to +32 is supposed to mean anyway.  High-
end Wacom tablets apparently give 3 angles: the rotation of the stylus, and 
two vertical angles w.r.t. the Wacom surface.  I think the orientation is 
supposed to be like the stylus rotation angle, and should really be on a per-
finger basis, so I guess we should call it rotation to be consistent, so that 
we are ready in case future touch devices report more than one angle as well.  
If the trackpad has a high density grid of capacitive sensors, then I think 
the idea is that trackpad's processor should be fitting an ellipse to each 
blob, and reporting the width, height and rotation of the ellipse.  (Or maybe 
the driver does it?)  If rotation was really not reported from each of the 
curve-fit ellipses, I wonder why.

The data coming from the BT trackpad looks like this

XI2 event type 19 seq 333 detail 400 pos 0x3D00000,0x28F0000 
976.000000,655.000000 root pos 2896.000000,679.000000
   valuator class label 636 value 71.000000 from range -2909.000000 -> 
3167.000000 name Abs MT Position X
   valuator class label 637 value -14.000000 from range -2456.000000 -> 
2565.000000 name Abs MT Position Y
   valuator class label 638 value 128.000000 from range 0.000000 -> 255.000000 
name Abs MT Touch Major
   valuator class label 639 value 168.000000 from range 0.000000 -> 255.000000 
name Abs MT Touch Minor
   tp  400  state  2  pos norm  QPointF(0.490454, 0.486357)  area  
QRectF(2886.81,666.686 18.3749x24.6289) angle 0  pressure  -1 

but with even larger values for the axes on the Macbook Pro.  So I decided to 
convert the value such that that maximum value corresponds to max finger width 
in mm (it's #defined to 10mm - perhaps a bit much) and convert that to pixels 
based on the actual screen resolution.

-- 
MVH, Shawn Rutledge ❖ "ecloud" on IRC