- Published on
World Labs turns one real-world robot task into thousands of simulated variations for training
- Authors
- Name
- DeepHorus
World Labs, the AI startup co-founded by renowned researcher Fei-Fei Li, has unveiled a new simulation engine designed to train robot control systems entirely in virtual environments. The company claims that models trained with this approach can then run reliably for hours on real hardware, addressing one of the biggest bottlenecks in robotics deployment. The technology, called "Real-to-Sim-to-Real" (R2S2R), comes from SceniX, a startup World Labs acquired in July.
The core problem, according to World Labs, is not model architecture but the sheer volume of experience a robot needs to operate reliably. Real-world data collection is expensive and difficult to control, and even online videos fail to systematically cover the full range of objects, physical conditions, and failure states. By turning one real-world task into thousands of controlled simulated variations, the engine aims to overcome this data scarcity hurdle.
The R2S2R engine captures robots, sensors, the environment, and task demonstrations, then rebuilds them as an interactive virtual world that behaves identically to the original. From a single real-world task, the system generates thousands of variations by altering lighting, object position and count, surrounding environments, physical properties like friction, and camera angles. To verify accuracy, World Labs runs identical action sequences in simulation and reality side by side, comparing observations, object movements, and outcomes.
Control models train in simulation and then transfer to real robots, with tests conducted on platforms like ALOHA, an open-source dual-arm design from Stanford. World Labs reports that models ran for one hour across four additional robot platforms without human intervention, handling tasks such as wrapping a power cord around a refrigerator and separating thin objects from a dense jumble. The system is not tied to a specific control model or robot type, allowing reconstructed worlds to be reused for new models and robots.
The company argues that simulation can effectively stand in for hardware during policy evaluation, even without perfectly matching real-world success rates. In tests using a two-handed cube handoff on an ALOHA robot, model rankings in simulation and reality remained largely consistent across different model types and training stages. Each checkpoint was evaluated using 2,000 simulated and 100 real runs, demonstrating that development teams can filter out weak model versions in simulation and save expensive hardware tests for the most promising candidates.
Industry observers see this as a significant step toward more scalable and cost-efficient robot learning. The simulator sits at the center of World Labs' broader world model strategy, potentially accelerating progress in physical AI. Looking ahead, the company plans to expand its simulation capabilities to more complex tasks and environments, while continuing to refine the fidelity of its virtual worlds. This development reflects a broader trend toward using simulation as a primary training ground for intelligent systems, promising to close the gap between simulation and real-world deployment.
