Claude AI Plays Pokémon on Twitch - Anthropic's AI Gaming

Anthropic's Claude 3.7 Sonnet Takes on Pokémon Red

Anthropic recently initiated a livestream on Twitch, titled "Claude Plays Pokémon," featuring its latest AI model, Claude 3.7 Sonnet, engaged in a game of Pokémon Red. This endeavor has evolved into a compelling experiment, demonstrating the current capabilities of artificial intelligence and observing public response.

AI and Video Game Benchmarks

Researchers in the field of AI have consistently utilized various video games, ranging from Street Fighter to Pictionary, as testing grounds for new models. Often, these applications are driven more by curiosity than practical necessity. However, Anthropic has indicated that Pokémon Red presented a valuable benchmark for Claude 3.7 Sonnet, enabling it to effectively process the puzzle-solving aspects inherent in the game.

Similar to models like OpenAI’s o3-mini and DeepSeek’s R1, Claude 3.7 Sonnet exhibits the capacity for “reasoning” through complex tasks, even those designed for a younger audience. In contrast to its predecessor, Claude 3.5 Sonnet, which encountered difficulties even leaving Pallet Town, Claude 3.7 Sonnet successfully obtained three gym leader badges.

Challenges and Observations

Despite its advancements, the newest Claude model still faces obstacles. During the livestream, the AI was halted by a rock wall, repeatedly attempting to pass through it without success.

A Twitch user humorously commented on the situation, questioning whether an AI with extensive programming would be defeated by a simple rock wall.

Ultimately, Claude was able to determine a path around the obstruction.

Watching Claude navigate Pokémon Red at a deliberate pace, carefully considering each action, can be both frustrating and strangely captivating. The stream displays Claude’s internal “thought process” alongside the real-time gameplay.

Navigating Social Interactions

At one point, Claude attempted to locate Professor Oak within his laboratory, but became confused by the presence of other non-player characters (NPCs).

“I have detected a new character positioned below me – an individual with dark hair and a white coat at coordinates (2, 10),” Claude articulated. “This may be Professor Oak! I will proceed to interact with them.”

The model then mistakenly engaged in conversation with a different NPC, one it had previously interacted with multiple times. This prompted some of the viewers in the Twitch chat to express their impatience, while others, having observed the stream for an extended period, remained more patient.

One viewer noted that the group had previously struggled with navigating the lab before understanding the correct procedure.

Echoes of Twitch History

The format of Anthropic’s stream may evoke a sense of nostalgia for long-time Twitch users. Over a decade ago, a groundbreaking online social experiment, Twitch Plays Pokémon, saw millions attempt to play Pokémon Red simultaneously. Control of the character was dictated by commands entered via Twitch chat, resulting in unpredictable and often chaotic gameplay.

This earlier event has served as inspiration for some AI researchers. In October 2023, Peter Whidden, a software engineer, published a YouTube video detailing his training of a reinforcement learning algorithm to play Pokémon. His AI spent over 50,000 hours in gameplay before achieving successful navigation, though it initially preferred observing the pixelated environment over actively playing.

From Collaboration to Spectatorship

While AI-driven “reenactments” like Whidden’s and Anthropic’s are entertaining, they also carry a hint of melancholy. The original Twitch Plays Pokémon was a defining moment in Twitch history, fostering a sense of community and shared purpose. Participants were united in their efforts to overcome challenges within the game.

Currently, it appears we are transitioning from collaborative experiences to more individual ones, observing an AI attempt a game that many of us mastered in childhood. This represents a broader trend: our online interactions are becoming increasingly solitary.