Hugging Face CSO Warns AI is Becoming 'Yes-Men on Servers'

A Cautious Perspective on AI's Scientific Potential

Founders of AI companies are often noted for expressing ambitious predictions regarding the technology’s capacity to transform various sectors, notably the scientific community. However, Thomas Wolf, the co-founder and chief science officer of Hugging Face, adopts a more tempered viewpoint.

The Risk of AI as Mere Conformity

In a recent essay shared on X (formerly Twitter), Wolf voiced concerns about AI devolving into “yes-men on servers” without a significant advancement in AI research. He clarified that current approaches to AI development are unlikely to produce systems capable of truly innovative and original problem-solving – the kind that typically leads to recognition like the Nobel Prize.

Wolf contends that a common misperception is to equate figures like Newton or Einstein with simply being exceptionally gifted students. He argues that genius isn’t merely a linear progression from high academic achievement.

The Need for Questioning, Not Just Answering

“To create an Einstein in a data center, we don’t just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask,” Wolf wrote. This highlights the importance of inquiry, not just information recall.

Wolf’s perspective differs from that of OpenAI CEO Sam Altman, who previously suggested that “superintelligent” AI could dramatically accelerate scientific breakthroughs. Similarly, Anthropic CEO Dario Amodei has forecasted AI’s potential to aid in discovering treatments for many forms of cancer.

Limitations in Knowledge Generation

Wolf’s central argument is that present-day AI systems don’t genuinely generate new knowledge through the connection of previously unrelated concepts. Despite having access to a vast amount of data, including much of the internet, AI primarily fills in gaps within the existing body of human knowledge.

This viewpoint is echoed by other AI experts, such as former Google engineer François Chollet, who believes that while AI can memorize reasoning patterns, it’s improbable that it can develop “new reasoning” in response to unfamiliar scenarios.

AI as Obedient Students, Not Revolutionaries

Wolf characterizes current AI labs as constructing essentially “very obedient students” – not the kind of groundbreaking scientific minds that drive progress. He explains that AI isn’t currently incentivized to challenge or propose ideas that contradict its training data, restricting its function to answering established questions.

He reiterates that creating an AI equivalent to Einstein requires a system capable of formulating questions that challenge conventional wisdom. Specifically, one that would consider “What if everyone is wrong about this?” even when faced with overwhelming consensus.

The Problem with AI Evaluation

Wolf attributes part of this issue to what he calls the “evaluation crisis” in AI. He points out that the benchmarks commonly used to assess AI system improvements typically involve questions with clear, straightforward, and “closed-ended” answers.

A Call for New Metrics

As a potential solution, Wolf advocates for a shift in the AI industry towards a measurement of knowledge and reasoning that can determine whether AI can adopt “bold counterfactual approaches,” formulate broad proposals from “tiny hints,” and pose “non-obvious questions” that open up “new research paths.”

He acknowledges that defining this new metric will be challenging, but believes the effort would be worthwhile.

The Value of Questioning Over Perfection

“[T]he most crucial aspect of science [is] the skill to ask the right questions and to challenge even what one has learned,” Wolf stated. “We don’t need an A+ [AI] student who can answer every question with general knowledge. We need a B student who sees and questions what everyone else missed.”