Two days after unveiling Solaris, an AI that generates software interfaces frame by frame instead of running code, Runway shipped something even stranger: GWM Worlds 2, a “General World Model” that generates a continuous, explorable environment in real time — 720p video at 24 fps with 48,000 Hz audio, responding to what you type and where you point the camera as you go, rather than producing a fixed clip from a prompt.
Not a video generator — a simulation you steer
Where a text-to-video model produces one finished clip per prompt, GWM Worlds 2 runs as an ongoing autoregressive stream: each frame is generated conditioned on what came before, using a structured “WorldPrompt” format that separates persistent world context (an environment description, subjects, physical rules, and a starting frame) from a timestamped stream of actions layered on top as you interact. Camera movement and subject actions can be issued independently, meaning you can walk, drive, or ride through a scene while separately directing what a character in it does — dialogue, gestures, object interactions — or reshape the environment itself with weather changes, lighting shifts, or object behavior, all through plain text.
Runway built it by fine-tuning its existing audio-visual generation model on the WorldPrompt format, then post-training the result into a real-time autoregressive system with separate video and audio decoders sharing a sliding-window cache of recently generated frames — the mechanism that lets the model stay responsive without regenerating its entire context on every new input.
Three ways to use it
Runway describes three interaction modes: authoring a full sequence of actions ahead of time and letting it render, a turn-based mode with explicit decision points between segments, and genuine real-time generation that responds to input as it arrives. The company’s own demos lean into the range that opens up — first- and third-person navigation through deserts, snowy landscapes, and city streets; a theater performance sequence; an AI agent independently navigating toward a goal or defending a position in a castle scenario; and a multiplayer mode where separate users control different subjects within the same generated world. Runway also points to continuing an existing video while keeping it visually consistent, and LLM-assisted “world authoring” for quickly setting up new scene presets, as practical entry points rather than novelty demos.
Where it’s headed, and where it still breaks
Runway names interactive entertainment, virtual characters, robotics, embodied-agent simulation, generative design, and interactive interfaces as the intended application areas — a considerably broader pitch than “video generation,” aimed more at simulation and agent training than content production alone. The company is direct about the current rough edges, too: quick camera rotations can still degrade textures and geometry, long-term memory of a scene is imperfect, there’s no way to reference an image beyond the starting frame or pre-fill video/audio into the stream, and free-form text control of things like NPC dialogue currently needs an external system to track world state, since the model itself has no persistent memory of who said what. Runway’s own framing is blunt about the stage this is at: “real-time video generation is still in its earliest stages, and the constraints outlined in this post will be solved with continued research.”