Every excavator operator learns the same unnatural translation early on: two joysticks, four axes, mapped in a way that has nothing to do with how an arm actually moves. MIT researchers led by Hermano Krebs in the Department of Mechanical Engineering, working with Sumitomo Heavy Industries, built a controller that skips the translation entirely — a miniature mechanical arm shaped like the excavator’s own boom and bucket, which the operator grips and simply moves the way they’d move their own arm to scoop something up.
Miming the motion instead of learning the mapping
The system, called the World-Space Interface (WSI), mirrors the operator’s physical gesture — reach, curl, lift — directly onto the machine’s world-space coordinates, rather than requiring the operator to mentally convert “push this stick this way” into “the bucket moves that way.” As Krebs put it, the goal is to let someone “mime picking up rocks or dirt, and the computer will do that translation to the world-space for us.” The WSI is paired with a six-screen simulation environment showing a digital excavator responding in real time to the operator’s arm movement, across a set of increasingly difficult virtual tasks — scooping, trenching, clearing debris, breaking rock.
In a seven-day study comparing the WSI against a standard joystick-based simulator, both novice and expert operators trained an hour a day across fifteen virtual environments. The results were published this month in the Journal of Computing in Civil Engineering, in a paper titled “A Comprehensive Training Platform for Excavator Operators: Training with Joint-Space and World-Space Interfaces in Virtual Environments.” Krebs’s team has also flagged haptic feedback as the next addition — giving operators resistance they can feel when the virtual bucket meets virtual dirt, not just a visual cue.
Why it fits here
This site covers gesture and embodied-control interfaces mostly through the lens of art and performance — mid-air gesture research, wearable EMG, radar-based touchless control. The WSI is the same underlying idea (translate a natural body movement into machine input, skip the abstract mapping) aimed at an unglamorous industrial task instead, and it’s a useful reminder that the interface problem this topic keeps returning to — how do you make a machine respond to a gesture the way a person actually expects it to — shows up just as sharply in construction equipment as it does in an interactive installation.
Related Reading
- MIT engineers design a better controller for operating construction diggers — MIT News
- New Controller Makes Heavy Machinery Intuitive — Hackaday
- A Comprehensive Training Platform for Excavator Operators — Journal of Computing in Civil Engineering
- Hermano Krebs — MIT Department of Mechanical Engineering