NVIDIA Cosmos: World Models & Infrastructure for Robotics

Nvidia's New AI Models for Robotics Development

On Monday, Nvidia introduced a suite of novel AI models, libraries, and infrastructure designed to empower robotics developers. The most prominent addition is Cosmos Reason, a 7-billion-parameter vision language model specifically engineered for reasoning within physical AI applications and robotic systems.

Expanding the Cosmos World Models

Alongside the launch of Cosmos Reason, Nvidia expanded its existing Cosmos world model family. This includes Cosmos Transfer-2, which facilitates accelerated synthetic data generation from 3D simulation environments or spatial control inputs.

A streamlined, speed-optimized version of Cosmos Transfer is also now available.

Synthetic Data for Enhanced Training

Nvidia highlighted during its announcement at the SIGGRAPH conference that these models are intended for the creation of synthetic datasets encompassing text, images, and video. These datasets are crucial for the effective training of robots and AI agents.

Cosmos Reason: Enabling Robotic Reasoning

According to Nvidia, Cosmos Reason equips robots and AI agents with the ability to “reason” through its integrated memory and understanding of physics.

This capability allows it to function as a planning model, predicting potential actions for embodied agents. The company indicates its utility in areas such as data curation, robotic planning, and video analytics.

New Neural Reconstruction Libraries

Nvidia also revealed new neural reconstruction libraries, including one supporting a rendering technique. This technique enables developers to simulate real-world scenarios in 3D utilizing sensor data.

This rendering functionality is being integrated into CARLA, a widely used open-source simulator for developers. An update to the Omniverse software development kit was also announced.

Robotics-Focused Servers

New server solutions tailored for robotics workflows have been introduced. The Nvidia RTX Pro Blackwell Server provides a unified architecture for robotic development workloads.

Furthermore, Nvidia DGX Cloud offers a cloud-based platform for streamlined management.

A Push into Robotics

These announcements signify Nvidia’s increasing investment in the robotics sector. The company views robotics as a key future application for its AI GPUs, extending beyond the current focus on AI data centers.

Feedback is valued! Help TechCrunch improve by completing this survey.