OpenAI Safety History Criticized by Former Policy Lead

Former OpenAI Researcher Criticizes Company's AI Deployment Narrative

Miles Brundage, a former policy researcher at OpenAI, recently voiced his concerns on social media regarding what he perceives as a revisionist account of the company’s approach to deploying potentially hazardous AI systems.

OpenAI's Current Stance on AI Safety

This week, OpenAI released a document detailing its present-day philosophy concerning AI safety and alignment – the process of creating AI systems that function predictably and in a manner aligned with human values.

The document outlines OpenAI’s view of Artificial General Intelligence (AGI), defined as AI capable of performing any intellectual task a human can, as an “ongoing progression.” This necessitates “repeated deployment and learning” from AI technologies.

The GPT-2 Precedent

OpenAI stated that, in the past, caution was exercised with systems like GPT-2, even if their capabilities appeared limited. They believe the development of AGI represents a series of increasingly useful systems, and learning from each iteration is key to safety.

However, Brundage contends that GPT-2 genuinely required significant caution upon its initial release, and that this approach aligns directly with OpenAI’s current iterative deployment strategy.

Brundage's Perspective on GPT-2's Release

“The release of GPT-2, in which I participated, was entirely consistent with, and actually predicted, OpenAI’s current philosophy of iterative deployment,” Brundage explained in a post on X.

He further noted that the model was released in stages, with insights shared at each step, and that numerous security professionals expressed gratitude for this measured approach.

Brundage's Role at OpenAI

Having joined OpenAI as a research scientist in 2018, Brundage eventually led the company’s policy research efforts. His work on the “AGI readiness” team specifically focused on the responsible deployment of language generation systems, including the ChatGPT chatbot platform.

GPT-2: A Foundation for ChatGPT

GPT-2, unveiled by OpenAI in 2019, served as a precursor to the AI systems that now power ChatGPT. It demonstrated the ability to answer questions, summarize text, and generate human-quality content.

Initial Concerns and Limited Release

Despite appearing rudimentary by today’s standards, GPT-2 was considered groundbreaking at the time. Due to concerns about potential misuse, OpenAI initially withheld the model’s source code, granting limited demo access to select news organizations.

Industry Debate Surrounding GPT-2

This decision sparked debate within the AI community. Some experts argued that the risks associated with GPT-2 were overstated and that there was no concrete evidence of potential abuse.

The Gradient, an AI-focused publication, even published an open letter urging OpenAI to release the model, asserting its technological importance.

Gradual Release and Brundage's Approval

OpenAI eventually released a partial version of GPT-2 six months after its initial announcement, followed by the complete system several months later. Brundage maintains that this phased release was the appropriate course of action.

“There was no aspect of the GPT-2 release motivated by viewing AGI as discontinuous,” he stated. He questioned whether the caution taken was truly “disproportionate,” suggesting that while a less cautious approach might have been acceptable in retrospect, it would have been irresponsible given the information available at the time.

Concerns About OpenAI's Current Approach

Brundage expresses apprehension that OpenAI’s recent document aims to establish a high threshold for demonstrating risk, requiring “overwhelming evidence of imminent dangers” before taking action. He considers this a “very dangerous” mindset for developing advanced AI.

“If I were still at OpenAI, I would inquire as to the rationale behind this document and what OpenAI intends to accomplish by downplaying caution so heavily,” Brundage added.

Historical Accusations and Competitive Pressures

OpenAI has previously faced accusations of prioritizing product launches over safety and of accelerating releases to gain a competitive edge. Last year, the AGI readiness team was disbanded, and several AI safety and policy researchers left the company.

Rising Competition in the AI Landscape

Competition in the AI field is intensifying. The Chinese AI lab DeepSeek gained recognition with its R1 model, which rivaled OpenAI’s o1 model on key performance indicators.

Sam Altman, CEO of OpenAI, has acknowledged that DeepSeek has narrowed OpenAI’s technological advantage and indicated that OpenAI will expedite its own releases to remain competitive.

Financial Implications and Trade-offs

Significant financial stakes are involved. OpenAI currently operates at a substantial loss, with projected annual losses potentially tripling to $14 billion by 2026. A faster release cycle could improve OpenAI’s financial performance in the short term, but potentially at the cost of long-term safety. Experts like Brundage question whether this trade-off is justifiable.