NVIDIA & Apple Vision Pro: Teaching Robots New Skills

Nvidia and the Advancement of Robotics

Robotics has proven to be a key driver of Nvidia's substantial expansion in recent years. The unveiling of GR00T in March of the previous year was widely recognized as a significant milestone in the field of humanoid robotics.

A large number of leading companies within the sector were granted access to GR00T at its initial release. These included prominent names such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, and Unitree Robotics.

Introducing GR00T Blueprint

During CES on Monday, Nvidia’s CEO, Jensen Huang, introduced GR00T Blueprint, a crucial addition to the GR00T platform. This new capability centers on imitation learning, a core technique for imparting new skills to robots.

This approach is particularly vital for humanoid robots, as their designs are intrinsically linked to the methods used for instruction.

The Principles of Imitation Learning

Imitation learning functions as its name suggests. A human demonstrator performs a task, and the robot replicates the observed actions.

This method is highly effective for training systems intended to automate pre-existing processes, particularly in settings like factories and warehouses, where the initial deployments of humanoid robots are currently taking place.

The Role of Teleoperation

Teleoperation plays a critical role in this learning process. It enables remote instruction of robots, effectively translating human actions into a digital format through a simulated environment.

Leveraging Apple’s Vision Pro

GR00T Blueprint allows users to record these actions utilizing Apple’s Vision Pro headset. The recorded performance is then converted into a digital twin.

This digital twin can subsequently be repeatedly practiced by the robot within a simulated environment, refining its performance and ensuring accurate replication of the desired task.