Eric Schmidt on AGI: Why a 'Manhattan Project' is a Bad Idea

A Call for Caution in AI Development

A recently published policy paper, authored by Eric Schmidt, former CEO of Google; Alexandr Wang, CEO of Scale AI; and Dan Hendrycks, Director of the Center for AI Safety, argues against a rapid, large-scale initiative to create AI systems possessing superhuman intelligence, often referred to as Artificial General Intelligence (AGI).

Potential Risks of an AI Arms Race

The paper, entitled “Superintelligence Strategy,” posits that an aggressive American effort to achieve exclusive control over superintelligent AI could provoke a strong response from China. This retaliation might manifest as a cyberattack, potentially destabilizing global political relationships.

The authors contend that assuming rivals will passively accept a lasting power imbalance is a flawed premise. They write that pursuing a “superweapon” and global dominance through AI carries the risk of escalating tensions and counter-measures, ultimately undermining the stability the strategy intends to achieve.

Challenging the "Manhattan Project" Analogy

This assessment comes from three prominent figures within the American AI sector, and directly challenges a proposal made by a U.S. congressional commission. This commission advocated for a “Manhattan Project-style” funding initiative for AGI development, mirroring the American atomic bomb program of the 1940s.

Recently, U.S. Secretary of Energy Chris Wright stated that the U.S. is embarking on a “new Manhattan Project” focused on AI, making this declaration alongside OpenAI co-founder Greg Brockman at a supercomputer facility.

The “Superintelligence Strategy” paper directly questions the prevailing belief, held by some American policymakers and industry leaders, that a government-funded AGI program is the optimal approach to competing with China.

Mutual Assured AI Malfunction (MAIM)

Schmidt, Wang, and Hendrycks suggest the current situation resembles a standoff akin to mutually assured destruction. Just as global powers refrain from seeking a monopoly on nuclear weapons – fearing a preemptive strike – they argue the U.S. should exercise caution in its pursuit of dominating extremely powerful AI systems.

The comparison of AI systems to nuclear weapons may appear drastic, but world leaders already recognize AI as a significant military advantage. The Pentagon has confirmed that AI is accelerating the military’s decision-making processes.

The authors introduce the concept of Mutual Assured AI Malfunction (MAIM), proposing that governments could proactively disable potentially threatening AI projects before adversaries can weaponize AGI.

Shifting Focus to Deterrence

Instead of striving to “win the race to superintelligence,” Schmidt, Wang, and Hendrycks advocate for a shift in focus towards developing strategies to deter other nations from creating superintelligent AI. They recommend expanding the government’s cyberattack capabilities to disable threatening AI projects controlled by other countries.

Furthermore, they suggest limiting adversaries’ access to both advanced AI chips and readily available open-source AI models.

Navigating Differing Perspectives on AI Risk

The co-authors identify two dominant viewpoints within the AI policy landscape. There are those who believe catastrophic outcomes from AI development are inevitable – the “doomers” – and advocate for slowing AI progress. Conversely, the “ostriches” believe nations should accelerate development, hoping for a positive outcome.

The paper proposes a third path: a balanced approach to AGI development that prioritizes defensive measures.

A Change in Perspective from Eric Schmidt

This strategy is particularly noteworthy coming from Eric Schmidt, who has previously emphasized the necessity of aggressive American competition with China in the development of advanced AI systems. He recently published an opinion piece asserting that DeepSeek represented a pivotal moment in the AI race between the two nations.

The Broader Geopolitical Context

While the Trump administration appears committed to advancing America’s AI development, the authors emphasize that U.S. decisions regarding AGI do not occur in isolation.

As the world observes America’s advancements in AI, Schmidt and his colleagues suggest that adopting a more defensive strategy may prove to be the more prudent course of action.