Research & Innovation

When Your Motion-Capture Rig Glitches Mid-Take, This Model Fixes It Before You Even See the Frame

REACT is a new motion-completion system built to patch occlusions and sensor noise in real time, frame by frame, with no look-ahead — the kind of fix that's normally done afterward, in post.

Anyone who’s worked with an affordable motion-capture rig knows the failure mode: a marker gets occluded for a few frames, a sensor drifts, high-frequency noise creeps in, and the resulting animation jitters or snaps unnaturally. The usual fix is a cleanup pass in post-production, after the whole take is already recorded. A new research framework called REACT — “Towards Real-Time Completion of Skeletal Motion Signals” — targets a harder version of that problem: fixing it live, frame by frame, as the data streams in, with no ability to look ahead at what comes next.

Why “causal” is the hard constraint

Most motion-cleanup techniques have the luxury of seeing the whole sequence before deciding how to patch a gap — if a hand disappears for ten frames, a system can use frames both before and after the gap to interpolate a plausible path. REACT works under the opposite constraint: it operates causally, producing a corrected estimate using only the frames it’s already seen, which is the actual requirement for anything running live — a game character being driven by a mocap suit, a VR avatar, a real-time interactive installation — where there is no “after” to peek at yet.

Built as an S4-based autoregressive model (a architecture designed for efficiently modeling long sequences), REACT works by coordinating multiple, complementary estimates of where a body part actually is, weighting them according to how reliable each estimate currently appears to be — the system becomes more cautious about a given limb’s tracked position the moment its own internal signals suggest that limb’s data is compromised by occlusion or noise. The researchers — a team split across InterDigital’s research labs and Inria’s Rennes group — evaluated REACT against several causal baselines and architectural variants, deliberately testing it under simulated occlusion and high-frequency noise at varying severity, and reporting the resulting inference latency alongside accuracy. Their conclusion is a tradeoff rather than a clean win: REACT lands at “a relevant compromise between reconstruction accuracy, update smoothness, and interactive inference” — not uniformly the most accurate option, but one that holds up across all three constraints simultaneously, which matters more for a live system than winning on accuracy alone while running too slow or jittering between updates.

Part of a longer research thread

REACT isn’t a one-off: the same core group — Philippe de Clermont Gallerande, Quentin Avril, Philippe Gosselin, Ferran Argelaguet, and Ludovic Hoyet — has been building toward this specific problem for at least a couple of years, including a 2025 paper on evaluating different body-part representations for motion reconstruction and a 2024 SIGGRAPH MIG poster on related motion-signal robustness. REACT reads as the culmination of that line of work: a full system-level contribution — a structured formulation of what “interactive motion completion” actually requires, plus a working instantiation of it — rather than an isolated benchmark result.