DeepSeek AI Model Outperforms OpenAI's GPT-1 on Reasoning Benchmarks

DeepSeek Releases Open-Source Reasoning Model, R1

DeepSeek, a Chinese AI laboratory, has made an open-source version of its reasoning model, DeepSeek-R1, publicly available. The company asserts that this model achieves performance levels comparable to OpenAI’s o1 on specific AI benchmarks.

Availability and Licensing

R1 is accessible through the AI development platform Hugging Face, released under a permissive MIT license. This licensing allows for unrestricted commercial utilization. DeepSeek reports that R1 surpasses o1 in performance on the AIME, MATH-500, and SWE-bench Verified benchmarks.

Benchmark Details

AIME utilizes other models to assess performance, while MATH-500 presents a series of mathematical word problems. SWE-bench Verified, conversely, concentrates on evaluating capabilities in programming tasks.

How Reasoning Models Differ

As a reasoning model, R1 incorporates a self-fact-checking mechanism. This feature mitigates common errors encountered in typical AI models.

While reasoning models generally require more processing time – typically extending solution times by seconds or minutes – they demonstrate increased reliability in fields like physics, science, and mathematics.

Model Parameters and Size

DeepSeek’s technical report indicates that R1 comprises 671 billion parameters. Generally, a higher parameter count correlates with enhanced problem-solving abilities.

Alongside the full-sized R1, DeepSeek has also released “distilled” versions, varying in size from 1.5 billion to 70 billion parameters. The smaller versions are capable of running on standard laptop computers.

Access to the complete R1 model is available via DeepSeek’s API, offered at a cost 90%-95% lower than OpenAI’s o1.

Community Adoption and Derivative Models

Clem Delangue, CEO of Hugging Face, announced on X (formerly Twitter) that developers have created over 500 “derivative” models based on R1. These derivatives have collectively amassed 2.5 million downloads – a figure five times greater than downloads of the official R1 model.

Limitations and Censorship

However, R1 is subject to scrutiny by China’s internet regulator, ensuring its responses align with “core socialist values.” Consequently, the model will not address sensitive topics such as the Tiananmen Square incident or the political status of Taiwan.

This censorship is common among Chinese AI systems, which often avoid responding to subjects that could provoke regulatory concerns, including discussions about the Xi Jinping regime.

Geopolitical Context

The release of R1 coincides with proposed stricter export controls and restrictions on AI technologies destined for Chinese entities by the Biden administration. Existing restrictions already prevent Chinese companies from acquiring advanced AI chips.

Proposed new regulations could impose tighter limitations on both semiconductor technology and the models required for developing sophisticated AI systems.

OpenAI’s Response and Concerns

OpenAI has advocated for U.S. government support for domestic AI development, warning that Chinese models could potentially equal or surpass U.S. capabilities. Chris Lehane, OpenAI’s VP of policy, specifically identified High Flyer Capital Management – DeepSeek’s parent company – as a source of concern.

Growing Chinese AI Capabilities

DeepSeek, Alibaba, and Kimi (owned by Moonshot AI) are among at least three Chinese labs that have unveiled models claiming to rival o1. DeepSeek was the first to announce a preview of R1 in late November.

AI researcher Dean Ball from George Mason University suggests this trend indicates Chinese AI labs will continue to rapidly adopt and improve upon existing technologies.

Ball also noted that the performance of DeepSeek’s distilled models will lead to the widespread availability of capable reasoning models, even on local hardware, potentially circumventing centralized control.

This article was originally published on January 20 and updated on January 27 with additional details.

Stay informed with TechCrunch’s AI newsletter! Subscribe here to receive it weekly in your inbox.