Gemini on Robots: Google's New AI Model Runs Locally

Gemini Robotics On-Device: A New Locally-Run Language Model

Google DeepMind unveiled a novel language model, Gemini Robotics On-Device, on Tuesday. This model is designed to execute tasks directly on robotic systems, eliminating the necessity for an internet connection.

This new iteration expands upon the capabilities of the previously released Gemini Robotics model from March. It empowers robots with the ability to manage their own movements.

Controlling Robots with Natural Language

Developers are now equipped to govern and refine the model’s performance through the use of natural language prompts, tailoring it to a diverse range of applications.

According to Google, benchmark testing indicates performance levels comparable to the cloud-based Gemini Robotics model. The company asserts its superiority over other on-device models in standard evaluations, although specific competitor models were not identified.

Demonstrations and Adaptability

A demonstration showcased robots powered by this local model successfully performing tasks such as unzipping containers and folding garments.

Initially trained for ALOHA robots, the model’s adaptability was further proven by its successful implementation on a bi-arm Franka FR3 robot and Apptronik’s Apollo humanoid robot.

The Franka FR3, in particular, demonstrated success in handling unfamiliar scenarios and objects. This included performing assembly operations on an industrial conveyor belt, even with items it had not previously encountered.

Gemini Robotics SDK and Training

Google DeepMind is also making available a Gemini Robotics SDK. This allows developers to train robots on new tasks by providing between 50 and 100 demonstrations, utilizing the MuJoCo physics simulator.

Growing Interest in Robotics AI

The integration of AI into robotics is gaining momentum across the industry. Several companies are actively exploring this field:

• Nvidia is developing a platform for creating foundational models specifically for humanoid robots.
• Hugging Face is involved in both the development of open-source models and datasets for robotics, as well as the construction of robots themselves.
• RLWRLD, a Korean startup backed by Mirae Asset, is focused on creating foundational models tailored for robotic applications.

These developments highlight the increasing investment and innovation within the realm of AI-powered robotics.