Framebuffer input-handling improvements

[tonyg]

Hi Eliot, all,

Working recently with a stripped-down Alpine installation on an oldish Thinkpad (X240) with a touchpad, I found the framebuffer input handling support did not work. The core issue is that /dev/input/event* for the touchpad yields ABS_* events, not REL_* events.

This branch makes a few changes to (a) very slightly clean up and (b) improve /dev/input handling in the framebuffer driver.

• Small sections of unused code removed.
• Option for flipping option/command, alt/meta, super/alt, etc (because it was strangely configured out-of-the-box for me)
• Crude (!) conversion of absolute multitouch input to relative, mouse-like input. Very simple tap detection, only deals with a single finger, not scroll wheel etc.

Potential improvements:

• Timeout for deciding a tap isn't happening, rather than solely being based on moving outside a fixed radius
• Configurable radius for motion hysteresis
• Handling of two-, three- and four-finger taps as left/middle/right mouse button; at present, the code just gives a single left click, not even a click-and-hold, so modifier keys are required for other mouse buttons
• Handling of Mac-style tap-and-hold for click-and-hold (should be fairly easy, requires interpretation of timestamps)
• more?

Ultimately it's a bit of a shame this is done in C; I've had good results reading /dev/input directly from Squeak (for the Squeakphone), so I'd be interested in discussing ways we could lift more of Squeak's event handling into the image. Until then of course it'd be nice to improve the C support code we have. Thoughts welcome!

[krono]

Oh thats neat!

[eliotmiranda]

Hi Tony,

well that’s great because there is a better way available to test. The threaded FFI VM (see the various [gcc3x-]cointerpmt.c files) is currently being used in anger at LabWare. There is a functional threaded ODBC connect and an in progress socket library. I believe both will be released open source.

So the possibility exists to interface directly to the event system using the FFI and callbacks since one can now make a blocking call for the next event in one thread while the VM continues to execute Smalltalk in other threads.

Currently build setups exist only for macOS. But it’s simple. One uses [gcc3x-]cointerpmt.c in place of [gcc3x-]cointerp.c and defines COGMT=1 on the c compiler command line. One would add buildmt, buildastmt & builddbgmt directories alongside build et al.

The basic scheme is that FFI calls block the entire vm until the next heartbeat (currently 500hz, but could be higher freq) at which time the vm spawns another thread to run Smalltalk if blocked in an FFI call. The FFI thread on its return path and any new thread race to control the vm. Above that one can bind Smalltalk processes to threads using the threadAffinity inst var. The VM’s thread scheduler will switch threads if necessary in process switch. There are a few more details.

So the threaded vm provides the necessary infrastructure to connect Smalltalk directly to the GUI subsystem.

[tonyg]

So the threaded vm provides the necessary infrastructure to connect Smalltalk directly to the GUI subsystem.

That is wonderful news! I'll look forward to it. However, the VM today is in good enough shape to allow the image to reach out and use /dev/input/event* on its own - see package LinuxIO at https://squeaksource.com/SqueakPhone/, which does just this.

This PR (still, I suppose, WIP) is instead intended to allow current Trunk images to work on framebuffer systems with a trackpad. For that, alterations to the C are both necessary and evil...