Camera-based hand tracking has become good enough that headsets ship without controllers. It also has a permanent blind spot: a camera can see where your fingers are, but not how hard they’re pressing. Pinching gently and squeezing hard look identical. For anything involving touch — playing an instrument, shaping material, grasping an object — that’s most of the information gone.
A paper submitted September 15 by researchers including several from Meta’s Reality Labs group (Svetoslav Kolev, Lingni Ma, Michael Goesele, Renzo De Nardi, Jakob Engel and Richard Newcombe) proposes a wristband that captures both.
How it works
The band carries flexible capacitive sensor arrays around the wrist, and critically requires no electrical contact with the skin — unlike EMG, which needs electrodes making good conductive contact and is sensitive to placement, sweat and skin condition.
The insight is anatomical. The muscles and tendons that move your fingers mostly live in your forearm, and they pass through the wrist. When you move or press with a finger, muscle contraction and tendon displacement push against the inside of the wrist in a distinctive pattern. Sense that pressure and you can infer what the hand is doing. A recurrent neural network maps the pressure signal to hand state.
To train and validate it, the team recorded synchronized wrist pressure, optical motion-capture hand pose and tactile-glove contact force, across isolated finger motion, fingertip force tests, and natural manipulation of everyday objects.
The results, honestly
- 4.6° mean finger-joint error for isolated single-user motion. That’s good.
- Contact force at R² = 0.57 across four users handling everyday objects, rising to 0.75 when an external pose signal is supplied.
An R² of 0.57 is a correlation, not a measurement. It captures the broad trend of how hard you’re pressing and not the precise value. The jump to 0.75 with external pose is the more revealing number: it says the wristband is best understood as one sensor among several rather than a complete solution, which is exactly how the authors frame it — one node in “a constellation of everyday wearables,” paired with something like an egocentric camera that supplies pose while the band supplies force.
Why creative technologists should watch this
Force is the expressive dimension that gesture interfaces threw away. Every acoustic instrument is played with it. Every material practice depends on it. Gesture control has been stuck at pointing, pinching and waving partly because those are the things a camera can see, and the results feel thin next to a keyboard that responds to how hard you hit it.
A wrist-worn band that adds a rough force channel to existing hand tracking is a plausible route to expressive control without gloves, which have never been comfortable enough to survive outside labs and studios. The no-skin-contact design is what makes it wearable all day, and being from a lab that ships wrist-worn hardware makes it more than a curiosity.
Not a product, and R²=0.57 is a long way from a velocity-sensitive key. But it’s the right axis of attack.