NVIDIA Omniverse Libraries Accelerate Fast, Reliable Robot Simulations

TL;DR

• NVIDIA announced updates to Omniverse libraries and Cosmos world foundation models (WFMs) at SIGGRAPH to speed and stabilize physically accurate robot simulations.
• OpenUSD serves as the core for physical AI workflows, enabling incorporation of 3D data and accelerating robotics, autonomous vehicles, and industrial systems development.
• OpenUSD Exchange SDK 2.0 adds UsdPhysics authoring and a layered asset structure to integrate physics directly into robot models; the SDK is open source and available on GitHub and PyPi.
• A SimReady materials library provides thousands of substrate materials for physical AI, supporting both visual and non-visual rendering for realistic sensor simulation in environments like Isaac Sim.
• Interoperability efforts with Google DeepMind enable MuJoCo (MJCF) and USD data exchange, including a mujoco-usd-converter for converting MJCF data to USD; DeepMind released an experimental USD import path into MuJoCo.
• NVIDIA’s NuRec libraries fuse RTX ray tracing with 3D Gaussian splatting for high-fidelity real-world sensor data reconstruction in OpenUSD environments, integrated with Omniverse Kit, Isaac Sim, and CARLA.
• Isaac Sim 5.0 and Isaac Lab 2.2 bring OpenUSD-based robot and OmniSensor schemas to standardize robotics data in USD; a new joint friction model within PhysX via OpenUSD schema improves simulation realism.
• Industry leaders such as Amazon Lab126, Boston Dynamics, Figure AI, Haply Robotics, Hexagon, and others are adopting these tools to accelerate robotics development.
• NVIDIA also promotes the OpenUSD Development certification to validate expertise in 3D content pipelines using OpenUSD. For more details, see the official NVIDIA blog covering the SIGGRAPH announcements. NVIDIA Omniverse blog

Context and background

At SIGGRAPH, NVIDIA highlighted a broad set of updates to the NVIDIA Omniverse libraries and Cosmos world foundation models (WFMs) that are designed to help developers build physically accurate virtual environments and AI agents capable of understanding and interacting with the real world. OpenUSD is positioned as the foundational technology for physical AI workflows, enabling the integration of 3D data for highly detailed simulations and faster robotics, autonomous vehicles, and industrial system development. This ecosystem aims to shorten iteration cycles, improve fidelity, and foster interoperability across major robotics and simulation platforms. The communications also underscore a broader push toward standardizing data formats and workflows so developers can move between simulation and real-world testing with lower friction [NVIDIA blog].

What’s new

• OpenUSD Exchange SDK 2.0: This update adds new modules for UsdPhysics authoring and introduces a layered asset structure to facilitate direct physics integration into robot models. It remains an open-source framework to streamline 3D data workflows and accelerate robotics development; availability is noted across GitHub and PyPi.
• SimReady materials library: An open-source collection of thousands of substrate materials authored in USD with MaterialX/OpenPBR features. Materials support both visual and non-visual rendering to enable accurate sensor simulation and physical interactions in simulation environments such as NVIDIA Isaac Sim, enabling faster iteration and higher fidelity in physical AI pipelines.
• MuJoCo and USD interoperability with Google DeepMind: A new mujoco-usd-converter enables interoperable conversion of MJCF data to USD using OpenUSD Exchange SDK 2.0 and MjcPhysics USD schemas. An experimental release from Google DeepMind enables USD data imports into MuJoCo via MjcPhysics USD Schemas and a new SdfFileFormatPlugin for integrating MJCF into USD, with native support for animation layers on the roadmap.
• NuRec libraries: NVIDIA’s NuRec combines RTX ray tracing with 3D Gaussian splatting to turn real-world sensor data into interactive OpenUSD simulation environments, delivering high-fidelity world reconstruction at scale. NuRec rendering is integrated in Omniverse Kit SDK, Isaac Sim, and CARLA.
• Isaac Sim 5.0 and Isaac Lab 2.2: Open-source robot simulation and learning frameworks; the early developer preview of Isaac Sim 5.0 includes OpenUSD-based robot and OmniSensor schemas for standardized robot and sensor definitions in USD. The PhysX extension introduces a new joint friction model defined via an OpenUSD schema, built in collaboration with Hexagon’s Robotics division and maxon.
• Industry adoption: Leading organizations, including Amazon Lab126, Boston Dynamics, Figure AI, Haply Robotics, Hexagon, Lightwheel, RAI Institute, Resim.ai, and Skild AI, are adopting NVIDIA Isaac libraries and AI models to accelerate robotics development.
• Tooling and certification: The ecosystem highlights Omniverse Kit SDK 108, SimReady materials, the MuJoCo-to-USD converter, USD Exchange SDK 2.0, and early previews of Isaac Sim 5.0 and Isaac Lab 2.2. NVIDIA also promotes the OpenUSD Development certification for validation of 3D content pipelines using OpenUSD.

Why it matters (impact for developers/enterprises)

The announced updates position OpenUSD as a practical, scalable backbone for physical AI workflows. By integrating physics directly into USD-based robot models and by enabling a richer library of materials and interoperable data formats, developers can achieve higher-fidelity simulations with faster iteration cycles. Cross-community interoperability with MuJoCo via the converter and DeepMind collaborations broadens data exchange pathways, reducing the gap between simulation and real-world deployment. Realistic sensor simulations enabled by SimReady materials and NuRec rendering help teams test perception, navigation, and control algorithms before pushing to real hardware, potentially lowering risk and speeding go-to-market timelines for robotics products and services. For enterprises, this translates into more reliable VR/robotics pipelines, better validation of AI agents, and more interoperable data pipelines across tools and platforms. These capabilities align with the broader industry need for standardized 3D data formats and scalable, high-fidelity simulation infrastructure.

Technical details or Implementation (highlights)

• OpenUSD Exchange SDK 2.0: Adds UsdPhysics authoring and a layered asset structure to embed physics into robot models. The SDK is open source and is accessible via repositories that host the project, supporting streamlined 3D data workflows and faster robotics development.
• SimReady materials library: An extensive open-source collection of substrate materials authored in USD with MaterialX/OpenPBR, designed to be importable into simulation environments like Isaac Sim and usable for both sensor and visual realism. This enables more accurate sensor simulation and realistic physical interactions, accelerating development cycles.
• MuJoCo to USD converter: A converter built with OpenUSD Exchange SDK 2.0 and MjcPhysics USD schemas to enable interoperable conversion of MJCF data to USD. An early release is available on GitHub and PyPi, with a Python CLI and a Python module interface for further transformations.
• Google DeepMind collaboration: An experimental path enables USD data imports into MuJoCo through MjcPhysics USD Schemas and a new SdfFileFormatPlugin to bring MJCF into the USD ecosystem. Native support for exporting simulation data as animation layers is anticipated.
• NuRec rendering: Combines RTX-based ray tracing with 3D Gaussian splatting to reconstruct real-world scenes as interactive OpenUSD environments. This rendering approach is integrated into the Omniverse Kit SDK, Isaac Sim, and CARLA, enabling scalable, high-fidelity world reconstructions.
• Isaac Sim 5.0 and Isaac Lab 2.2: Open-source robot simulation and learning frameworks featuring OpenUSD-based robot and OmniSensor schemas for standardized robot/sensor definitions in USD. The PhysX extension includes a new joint friction model defined in OpenUSD, enabling more realistic robot motion when simulated against physics.
• Industry adoption and ecosystem: Organizations across the robotics and AI sectors are adopting these tools to accelerate development, validate AI policies, and improve integration across simulation and real-world deployments.
• OpenUSD ecosystem and certification: NVIDIA highlights ongoing development with OpenUSD, including a certification program to validate expertise in building and maintaining USD-based pipelines.

Key takeaways

• OpenUSD is positioned as the core for physical AI workflows, enabling robust, scalable simulations.
• The OpenUSD Exchange SDK 2.0 expands physics authoring and asset structuring for robot models.
• A SimReady materials library enhances material realism and sensor fidelity for faster, more accurate simulations.
• Interoperability with MuJoCo via the mujoco-usd-converter broadens data exchange possibilities between MuJoCo and USD.
• NuRec rendering provides high-fidelity world reconstruction within Omniverse, Isaac Sim, and CARLA.
• Isaac Sim 5.0 and Isaac Lab 2.2 standardize robot and OmniSensor definitions in USD and introduce improved physics realism via a new Joint Friction model in PhysX.
• Industry leaders are actively adopting these tools, signaling broader enterprise impact.
• OpenUSD Development certification offers a credential for professionals building USD-based pipelines.

References

• https://developer.nvidia.com/blog/developers-build-fast-and-reliable-robot-simulations-with-nVIDIA-omniverse-libraries