- Published on
How to Use AI Agents to Prepare 3D Scenes for Simulation
- Authors
- Name
- DeepHorus
NVIDIA published a technical blog post on September 16, 2026, by Max Bickley and Ashley Goldstein detailing how agentic AI workflows can prepare 3D scenes for robotics simulation. The workflow uses NVIDIA Omniverse Libraries to transform a Blender scene into a simulation-ready OpenUSD handoff for NVIDIA Isaac Sim or NVIDIA Isaac Lab. It combines reasoning, tool execution, and validation in a repeatable process designed to remove a common bottleneck in physical AI development.
Robotics teams often blame the policy, model, or training loop when projects stall, but the blog argues the bottleneck frequently appears earlier: the robot has no simulation-ready world. A 3D artist may have created the assets in Blender, yet the scene still needs semantic labels, correct collision meshes, simulation-aware materials, configured sensors, clean USD export, and validation. That preparation work is tedious, repetitive, time-consuming, and easy to get wrong. It is also often out of scope for the robotics simulation engineer.
The proposed workflow assigns coordination to Codex, powered by OpenAI GPT-6 Astra, which interprets results and guides iteration. Specialized subagents built with the Hermes agent harness and deployed through NVIDIA NemoClaw use Omniverse Libraries to inspect and modify the scene. These agents rely on tools such as ovphysx for physics authoring, ovrtx for visual preflight rendering, and SimReady validation against target profiles. OpenUSD acts as the shared contract layer, preserving hierarchy and metadata so agents can iteratively author labels, collision meshes, rigid-body properties, and sensor definitions without flattening the original creative work.
In practice, the workflow inspects the scene through a Blender Model Context Protocol server and then authors USD data. Agents tag prims with task-relevant classes, configure camera and lidar sensors, and apply physics through ovphysx. They render review views using ovrtx and run SimReady validation as the acceptance gate before handoff. Only after passing that validation is the scene delivered to Isaac Sim or Isaac Lab.
The blog emphasizes that simulation preparation determines whether a robot can perceive target objects and whether the scene will export cleanly to USD. Without such checks, engineers risk burning time debugging issues in Isaac Sim or Isaac Lab. By using agents with the right tools, teams can automate repetitive authoring and validation while keeping the scene hierarchy intact. This approach makes scene preparation a first-class part of the physical AI pipeline rather than an afterthought.
NVIDIA positions the workflow as an example of what agentic systems should handle when they have access to specialized tools. The post notes that Codex or Claude can coordinate the overall task, while NemoClaw deploys Hermes subagents for specific functions. It also directs developers to the Omniverse Labs GitHub repository for agent-callable samples across USD, rendering, physics, storage, and validation. The SimReady Foundation is cited for understanding validation profiles and simulation-ready USD requirements.
Looking ahead, the workflow could accelerate digital twin creation and physical AI training by making simulation-ready worlds easier to produce. As agentic AI matures, scene inspection, physics authoring, sensor configuration, and validation may become increasingly automated. The blog suggests that repeatable handoffs to Isaac Sim and Isaac Lab can help robotics teams focus on policy and training rather than manual scene cleanup. NVIDIA's next steps invite developers to explore Omniverse Labs, SimReady Foundation, and NemoClaw to build specialized agents for robotics scene preparation.
