- Published on
How to Use AI Agents to Prepare 3D Scenes for Simulation
- Authors
- Name
- DeepHorus
NVIDIA has introduced an agentic AI workflow that prepares 3D scenes for robotics simulation, converting Blender assets into simulation-ready OpenUSD worlds for Isaac Sim and Isaac Lab. The workflow uses Codex, powered by OpenAI GPT-6 Astra, as the main coordinator, with specialized subagents built on the Hermes agent harness and deployed through NVIDIA NemoClaw. These agents inspect scenes, add physics properties, render preflight views, and validate results against SimReady requirements.
The challenge is that many robotics projects stall not at the policy, model, or training loop, but earlier: the robot does not have a simulation-ready world to train in. Manual prep work—labeling objects, correcting collision meshes, configuring sensors, and ensuring clean USD export—is tedious, repetitive, and easy to get wrong. Agentic systems with the right tools can automate this work.
A general-purpose agent like Codex or Claude Cowork can recognize that a Blender scene needs simulation readiness, but recognition alone is not enough. The agent must act: inspect the scene, identify missing simulation metadata, add semantic labels, configure sensors, author collision shapes and physics properties, render visual preflight views, run SimReady validation, automatically repair safe issues, and escalate ambiguous decisions to a human.
The workflow turns a broad request into a multi-agent engineering process. Codex uses Astra to translate the developer’s objective into tasks, identify dependencies, and review results from Hermes subagents deployed through NemoClaw. For example, making an object grabbable requires coordinated updates to its semantic label, rigid-body configuration, and collision geometry.
NVIDIA Omniverse Libraries provide the tools that subagents call to act on the scene. OpenUSD serves as the interchange format, connecting reasoning, tool execution, and validation into a repeatable process for delivering a simulation-ready world. This reduces the bottleneck that often prevents robots from training effectively in simulation.
The approach reflects a broader shift toward agentic AI in physical AI development. By automating scene preparation, developers can focus on policy design and training rather than manual 3D cleanup. The workflow is designed to be repeatable and scalable across robotics teams, making simulation-ready worlds more accessible.
