OpenAI O3 Model Costs: Higher Than Expected?

OpenAI's o3 Model: Revised Cost Estimates

When OpenAI introduced its o3 “reasoning” AI model in December, a collaboration with the developers of ARC-AGI, a benchmark for evaluating advanced AI, was initiated to demonstrate o3’s potential. Subsequent to this, the initial results have undergone revision, presenting a somewhat altered picture of the model’s performance.

Updated Cost Assessments

The Arc Prize Foundation, responsible for maintaining and administering ARC-AGI, recently updated its calculations regarding the approximate computing expenses associated with o3. Initially, the organization estimated that the highest-performing o3 configuration, designated 'o3 high', required approximately $3,000 to resolve a single ARC-AGI challenge. However, the Arc Prize Foundation now believes this cost is significantly higher – potentially reaching $30,000 per task.

Implications for AI Costs

This revision is significant as it highlights the potential expense of utilizing today’s most advanced AI models for specific tasks, particularly in their early stages of deployment. OpenAI has not yet announced pricing for o3, nor has the model been publicly released. Nevertheless, the Arc Prize Foundation considers OpenAI’s o1-pro model pricing to be a suitable benchmark for comparison.

For reference, o1-pro currently represents OpenAI’s most expensive model available.

Rationale Behind the Revision

“We believe o1-pro offers a more accurate comparison of the true cost of o3… considering the substantial compute time utilized during testing,” explained Mike Knoop, a co-founder of the Arc Prize Foundation, in a statement to TechCrunch. “However, this remains an approximation, and we have maintained a ‘preview’ label for o3 on our leaderboard to acknowledge this uncertainty until official pricing is disclosed.”

Computational Resource Usage

A higher price point for o3 high would align with expectations, given the model’s reported consumption of computing resources. The Arc Prize Foundation indicates that o3 high required 172 times more computing power than o3 low, the least resource-intensive o3 configuration, when addressing ARC-AGI tasks.

Potential Enterprise Pricing

Furthermore, speculation has been circulating regarding premium pricing plans that OpenAI is contemplating for its enterprise clientele. Reports from The Information in early March suggest the company may introduce charges of up to $20,000 monthly for specialized AI “agents,” such as a dedicated software development agent.

Efficiency Considerations

While some may contend that even OpenAI’s most expensive models will remain more cost-effective than employing a typical human contractor or staff member, AI researcher Toby Ord suggests that efficiency may be a factor. For instance, o3 high necessitated 1,024 attempts per task within ARC-AGI to achieve its optimal score.

• The revised cost estimate for o3 high is now around $30,000 per task.
• o3 high used 172x more computing power than o3 low.
• OpenAI may charge up to $20,000 per month for specialized AI agents.