NVIDIA published a set of internal case studies on November 5 showing technical artists and engineers directing AI coding agents — Claude and NVIDIA’s own GPT-6 Astra — through its Omniverse libraries to build simulation-ready 3D content. The common thread across all of them: the agent writes and runs code, but the output lands in the same editable formats (OpenUSD, Houdini scenes) an artist would normally hand-build, so a human can still open it up and change it.
Recreating a synth from product photos
The most concretely “creative tech” example comes from Nick Scarcella, a senior technical artist on NVIDIA’s BondLab team. Scarcella directed Astra to recreate a Teenage Engineering OP-1 portable synthesizer in 3D, working from product images, manuals, and his own technical direction, with Astra and Claude’s Opus model doing successive passes to refine details. He exported the result into OpenUSD, tested it in Houdini and Blender, then used NVIDIA’s ovrtx library to develop materials, lighting, and the final rendered look.
The same team used a separate, publicly shared robot-hand checkpoint in Isaac Sim, directing its motion and lighting with Claude’s help to build an “Echoes” visualization — translucent overlapping hand poses merged into a single animation, meant as a visual read on how a robot-training run actually moves.
Beyond the art pipeline
The other case studies push the same agent-plus-Omniverse pattern into engineering territory: a senior OpenUSD product manager directed Astra to model a car suspension in PTC Onshape and configure a disassembly test in Isaac Sim, using simulation feedback to redesign a wrench that could actually reach the bolts. Another engineer used Astra to turn panoramic site-scan photos into an editable, empty building model for warehouse-automation planning, aligning the generated geometry against point-cloud measurements.
NVIDIA frames all of this as “agent-assisted creation into existing artist tools” rather than agents replacing the tools outright — the output in every case is something a person can still open, inspect, and hand-edit in Houdini, Blender, or Onshape, not a black-box render.
Why this matters for creative coding specifically
What separates this from a generic “AI writes code” story is that the agent’s output is explicitly scoped to tools a 3D artist already uses daily — geometry nodes, USD stages, material graphs — rather than a standalone prompt-to-render pipeline that bypasses them. For creative coders, that’s the more interesting development: the agent is acting as a collaborator inside Houdini’s own node graph, not a replacement for it, which keeps the artist’s usual iteration loop (inspect, tweak, re-render) intact.
Related Reading
- New NVIDIA Cosmos 3 Open Models for Interactive World Simulation
- NVIDIA’s SIGGRAPH Keynote: DLSS 5 Goes Multi-Model, and Cosmos 3 Pushes World Models to the Edge
- Into the Omniverse: How Developers Turn Ideas Into Simulations With Frontier AI Agents — NVIDIA
- Scarcella’s Houdini MCP tools and agent skill — GitHub
- Robotics startup Sensori’s SimReady Publisher story — Onshape