Australian director, animator and video artist Tee Ken Ng has built Plato: a projected platformer in which you play a small neon-green square collecting coins, and you build the level by putting physical objects in front of the projector. Their shadows become the ground.
The pipeline is four pieces and no custom hardware
- A webcam registers the shadows
- TouchDesigner converts them into low-resolution collision surfaces in real time
- Godot turns those surfaces into interactive physics geometry, so the square can run and roll on them, and jump off or over them
- MadMapper handles the projection mapping
And the alignment, in the artist’s words:
“The camera, projector, and game are mapped to the same space, so the collision map lines up with the shadows you see on the wall.”
That single sentence is the entire technical problem and the entire reason it works. Three devices with three different coordinate systems — a camera’s image plane, a projector’s image plane, and a game’s world space — have to agree closely enough that the ground you see is the ground the physics uses. If they disagree by a few centimetres, the square floats above the shadow or sinks into it, and the illusion that you are building the level is gone. Get them aligned and the shadow is the level, with no explanation required.
The low resolution is a feature rather than a limitation. Collision geometry derived from a camera at full resolution would be noisy, expensive, and would produce a surface the physics engine fights with. Coarse surfaces are stable, cheap, and match the chunky aesthetic of a neon square on a wall.
Anybody can build this
Ng notes that in principle anyone with a projector, a camera, and MadMapper and TouchDesigner running can play it. And anything can become a collision object — including moving things like your own hands, though those come with a little lag.
The lag detail is honest and worth keeping. The chain is camera capture → TouchDesigner processing → surface extraction → Godot physics → render → projector, and every stage adds latency. For a static object that costs nothing. For a hand you are actively moving, the geometry trails your actual hand by enough to notice. A piece that leans on hand movement would feel broken; one built around placing objects and then playing does not.
The title is doing work
Plato is a reference to the allegory of the cave — prisoners who see only shadows projected on a wall and take them for the truest form of reality, because it is the only reality available to them.
So the neon square is a prisoner of a reality assembled from a pile of random objects and a lamp. It is a neat joke, and it is also a reasonably precise description of the mechanic: the square’s physics are genuinely and completely determined by the shadows. It has no access to the objects, only to their projections, and the projections are its world.
Plenty of work takes its title from a philosophical reference and gets nothing from it. Here the allegory and the implementation are the same structure, which is rarer.
Why it belongs in the interface conversation
Camera-and-projector setups are among the oldest ideas in interactive installation, and most of them use the camera to find people — blob tracking, skeleton tracking, hand detection. This one uses the camera to find absence of light, which is a far more robust signal than any of those, needs no model, works on anything opaque regardless of shape or colour, and degrades gracefully.
It also means the interface has no vocabulary to learn. There are no gestures. You put a thing somewhere and the thing is now part of the game. The affordance is “objects block light,” which every visitor already knows.
Ng’s other work uses light and shade as its medium across video direction and game design, and is worth looking at on that basis.