Tesla Doubles Down on Vision-Only Autonomy with Supercomputer Power

Tesla Advances Autonomous Driving with New Supercomputer

Elon Musk, CEO of Tesla, has discussed the development of a neural network training computer, dubbed “Dojo,” since 2019. Musk envisions Dojo as a key component in achieving fully autonomous driving through vision processing. Currently, while Dojo remains under development, Tesla has unveiled a new supercomputer designed to function as a prototype, showcasing the capabilities Dojo aims to deliver.

Radar and Lidar Alternatives

During the 2021 Conference on Computer Vision and Pattern Recognition, Andrej Karpathy, Tesla’s head of AI, presented the company’s latest supercomputer. This system allows Tesla to move towards eliminating the need for radar and lidar sensors in self-driving vehicles, prioritizing high-resolution optical cameras instead.

Karpathy emphasized that achieving human-like responses from a computer in novel environments necessitates extensive datasets and a supercomputer with substantial processing power. This power is crucial for training the neural networks that underpin Tesla’s autonomous driving technology, driving the creation of these Dojo precursors.

Supercomputer Specifications

Tesla’s newest supercomputer boasts 10 petabytes of high-speed NVME storage and operates at a rate of 1.6 terabytes per second, as detailed by Karpathy. He estimates its performance at 1.8 EFLOPS, potentially ranking it as the fifth most powerful supercomputer globally.

However, Karpathy clarified that this ranking is preliminary, as the team has not yet completed the necessary benchmark testing for inclusion in the TOP500 Supercomputing list. He noted that its computational capacity is comparable to Nvidia’s Selene cluster, which currently holds the fifth position.

Vision-Only Autonomy

Musk has consistently championed a vision-only approach to autonomous driving, largely because cameras offer faster data acquisition compared to radar or lidar. As of May, Tesla Model Y and Model 3 vehicles manufactured in North America are being produced without radar, relying solely on cameras and machine learning for advanced driver assistance features.

Unlike many autonomous driving companies that depend on lidar and detailed high-definition maps, Tesla aims for a system that can function universally. These other systems require comprehensive maps detailing road layouts, traffic signals, and other critical infrastructure.

“Our strategy centers on vision-based systems, primarily utilizing neural networks capable of operating anywhere on Earth,” Karpathy explained during his workshop.

Benefits of a Silicon Computer

Replacing a human driver with a silicon-based computer offers several advantages, including reduced latency, a 360-degree awareness of surroundings, and unwavering attention, according to Karpathy. This eliminates distractions like checking social media.

Real-World Applications

Karpathy showcased scenarios where Tesla’s supercomputer utilizes computer vision to improve driver safety. These include emergency braking systems that detect pedestrians and traffic alerts that warn drivers about upcoming yellow lights.

Tesla vehicles have also implemented a feature called pedal misapplication mitigation. This system identifies pedestrians or obstacles in the vehicle’s path and intervenes if the driver accidentally presses the accelerator instead of the brake, potentially preventing accidents.

Data Acquisition and Processing

The Tesla supercomputer processes video feeds from eight surrounding cameras at 36 frames per second, generating a substantial amount of environmental data, Karpathy detailed.

While the vision-only approach offers scalability advantages over maintaining high-definition maps, it presents significant challenges. The neural networks must process vast quantities of data at speeds comparable to human perception and reaction times.

Supervised Learning and Auto-Labeling

Karpathy believes this challenge can be overcome through supervised learning. Testing has shown that the technology can operate with zero interventions in sparsely populated areas, but struggles in complex environments like San Francisco.

A key innovation for the Tesla AI team has been auto-labeling. This technology automatically identifies and labels objects, such as roadway hazards, from millions of videos captured by Tesla vehicles. Traditionally, large AI datasets require extensive manual labeling, a time-consuming process.

Data Volume and Future Development

With the latest supercomputer, Tesla has amassed 1 million videos, each approximately 10 seconds long, and labeled 6 billion objects with depth, velocity, and acceleration. This data occupies 1.5 petabytes of storage.

Despite this substantial volume, further data and more powerful supercomputers are needed to achieve the reliability required for a fully autonomous driving system reliant on vision alone. Tesla continues to invest in developing increasingly advanced AI capabilities.