Nethack and AI: How a Retro Game Could Predict the Future

AI Tackles the Ultimate Gaming Challenge: NetHack

Machine learning models have demonstrated proficiency in games like Chess, Go, and Atari, but researchers at Facebook are setting a new, more demanding goal: for AI to conquer NetHack, a game renowned for its difficulty and complexity.

Edward Grefenstette of Facebook AI Research stated, “We aimed to establish a highly accessible ‘grand challenge’ with this game. It won’t definitively solve AI, but it will pave the way for improved AI.” He further explained that games serve as valuable testing grounds for challenging assumptions about machine intelligence.

Understanding NetHack's Influence

NetHack is a profoundly influential game in the history of gaming. Players assume the role of an adventurer exploring a constantly changing dungeon filled with monsters, traps, and other perils. Success requires strategic combat, careful navigation, and maintaining favor with a deity.

It is considered the first “roguelike” game, evolving from its predecessor, Rogue, and remains arguably the most challenging in the genre. The game is freely available for download and playable on a wide range of platforms.

Despite its simple ASCII graphics – utilizing characters like 'g' for goblins and '@' for the player – NetHack possesses incredible depth. Since its debut in 1987, a dedicated team of developers has continually expanded the game with new objects, creatures, rules, and interactions.

This continuous development contributes to NetHack’s complexity and makes it a unique challenge for AI. The game’s open-ended nature allows for countless interactions between objects and creatures, most of which are meticulously hand-coded.

NetHack with a tile-based graphics update – all the information is still available via text.

Why NetHack Differs from Other AI Challenges

“Solutions developed for games like Atari, Dota 2, and StarCraft 2 are insightful, but NetHack presents distinct hurdles. It necessitates leveraging human knowledge to play effectively,” Grefenstette noted.

Unlike other games with relatively clear winning strategies, NetHack is inherently unpredictable. While games like Dota 2 have defined objectives and controllable pieces, NetHack introduces a dynamic world that changes with each playthrough.

“Each dungeon is unique, with new monsters and items. There are no save points; death means starting over. It mirrors aspects of real life,” Grefenstette explained. “Learning from mistakes and applying that knowledge to new situations is crucial.”

Simple actions, like drinking a potion, can have unexpected consequences. An AI must consider not only the direct effect but also potential alternative uses, such as throwing it at an enemy or using it to unlock a chest. This requires a level of intuitive reasoning that doesn’t come naturally to AI.

The game’s intricate systems are difficult to fully articulate, but this complexity makes it an ideal platform for a competition, according to Grefenstette. “Playing the game requires human-like understanding,” he emphasized.

The NetHack Learning Environment

For years, developers have created NetHack bots using decision trees. The Facebook Research team aims to inspire a new approach by creating a training environment for machine learning-based game-playing algorithms.

NetHack screens with labels showing what the AI is aware of.

The NetHack Learning Environment (NLE) was established last year, providing a dedicated computing environment where an AI can interact with the game through text commands. This allows for interaction via directions and actions like attacking or consuming items.

NetHack presents a compelling target for AI designers. While games like StarCraft 2 may have greater visibility, NetHack’s legendary status and the opportunity to develop AI on fundamentally different principles are particularly appealing.

Furthermore, NetHack is more accessible than many other AI challenges. Unlike StarCraft 2, which requires significant computing power for visual recognition, NetHack transmits all game information via text, making it exceptionally efficient to process. It can be simulated thousands of times faster than a human could play.

“Our goal was to create a research environment that posed significant challenges for the AI community without limiting participation to large academic labs,” Grefenstette stated.

The NetHack Challenge

For the coming months, the NLE will be available for testing. Competitors can develop their bots using any method they choose. However, during the official competition, starting October 15, interaction with the game will be restricted to standard commands, preventing access to internal game data.

The competition’s objective is to complete the game, with the Facebook team tracking the number of times an agent “ascends.” However, Grefenstette conceded that achieving ascension is unlikely for any participant, given the game’s extreme difficulty. Additional scoring metrics will be used to determine winners in various categories.

The ultimate hope is that this challenge will foster a new approach to AI, one that more closely mimics human thought processes. Techniques like shortcuts, exploitation, and brute force will be ineffective; the AI must learn logical systems and apply them intelligently to survive encounters with formidable creatures.

Detailed rules and information about the NetHack Challenge can be found here. The results will be announced at the NeurIPS conference later this year.