Unfinished Motion, a project by Hsiao Yung-Shen presented through the IRCAM Forum Workshops Hors-les-Murs Tokyo 2026 collection, builds a real-time system that translates a pipa performer’s physical gestures — plucking force, contact angle, subtle wrist rotations — into a generative soundfield and particle visuals. The project’s framing is specific: “the body is not just the source of sound, but the intention of sound and space.”
Reading effort, not just position
The pipa is a traditional Eastern plucked-string instrument, and its performance technique carries expressive information well beyond which notes get played — the same note struck with different plucking force or wrist angle at the technical level means something different musically. Unfinished Motion uses Laban Movement Analysis, a framework for quantifying qualities of physical movement (called “Effort” in Laban’s terms), to capture that expressive layer rather than discarding it.
The pipeline
Four tools chain together into what the project calls a low-latency “Body–Digital–Sound–Vision” pipeline:
- MediaPipe handles pose recognition, tracking the performer’s body and hands.
- Dicy2, a machine-learning tool for co-creative dialogue, generates responsive material rather than fixed playback.
- Spat5 handles multidimensional spatial sound, placing and moving audio through a room rather than a fixed stereo field.
- TouchDesigner renders the particle visuals in response to the same movement data.
Still in progress, deliberately
The project’s evaluation plan reflects its “Unfinished Motion” name: audience perceptual data collection happens in two stages, starting with a preliminary symposium at a December 2026 pilot showcase, followed by a fuller presentation with live perceptual testing in late 2027. It’s a research project mid-development, not a finished installation — worth tracking as it moves toward its full presentation.