Research & Innovation

DyCraft Teaches AI Agents to Animate Like Stop-Motion Directors, Not Physics Engines

Coding agents can already build decent 3D scenes and assets, but long animated sequences come out with interpenetrating objects and physically impossible motion. DyCraft fixes this by making the agent storyboard key poses first, verify each one is physically valid, and only then fill in the motion between them.

Give a modern coding agent a single static scene to build in 3D, and it can usually do a respectable job — placing objects, texturing surfaces, getting proportions right. Ask the same agent to animate a long, multi-shot sequence end to end, and the results tend to fall apart in a specific, recognizable way: objects drift into each other, things that should rest on a surface float in midair, and motion that should look caused by physics instead looks arbitrary. A new system called DyCraft, from researchers including Jeff Tan, Hang Dai, and Stanford’s Jiajun Wu, diagnoses that failure mode precisely and proposes a fix borrowed from an old craft rather than a new algorithm.

The problem with asking an agent to animate everything at once

Coding agents are good at building and debugging things they can verify in isolation — a static scene either renders correctly or it doesn’t, and the agent can check. Long-horizon animation breaks that feedback loop: there’s no simple, local check for “is this 40-shot sequence of motion physically coherent,” so an agent generating it end to end has no good signal telling it when things have gone wrong until the final result is already a mess of clipping geometry and unnatural movement.

Storyboard first, like a human animator would

DyCraft’s core idea is to split animation generation the way a human animator actually works: first build a static scene, then construct a 3D storyboard of key poses — essentially snapshots of the important moments in the sequence — where each individual pose is itself just a static scene the agent can build and verify in isolation, the same way it already handles single-scene generation well. Only after those key poses are confirmed physically valid does the system hand the problem of filling in the motion between them to dedicated motion-authoring tools, rather than asking one monolithic process to solve composition, physics, and motion simultaneously.

That decomposition turns an unverifiable, holistic problem into a sequence of smaller, checkable ones — verified start states, verified end states, and a constrained motion-generation step in between. It’s a structural echo of how actual stop-motion and keyframe animators have always worked: block the poses that matter, then animate the transitions.

How it was tested

The researchers evaluated DyCraft on 24 long-horizon, multi-shot tasks spanning humanoid robots, human and animal characters, and physical mechanisms — a deliberately broad test set rather than one narrow animation style. Compared against both a native agent running inside a coding environment (the paper references “GPT-6 Astra in Codex” as a baseline) and prior agentic animation approaches, DyCraft reduced physical-violation rates and was preferred more often by human evaluators judging the output.

The paper, posted to arXiv on October 29 with a project page live at dy-craft.github.io, carries no journal reference yet — an unreviewed preprint, though its evaluation methodology and the specific failure mode it targets are concrete enough to be independently checked against the examples on its own project site.