GPT-4.1: OpenAI's New AI Models Focus on Coding

OpenAI Introduces the GPT-4.1 Model Family

On Monday, OpenAI announced the release of a new series of models designated GPT-4.1. This launch introduces a tiered system, with variations including GPT-4.1, GPT-4.1 mini, and GPT-4.1 nano, potentially adding to the existing complexity of the company’s naming conventions.

Enhanced Capabilities for Coding and Instruction

OpenAI states that all three models – GPT-4.1, GPT-4.1 mini, and GPT-4.1 nano – demonstrate significant proficiency in both coding tasks and following instructions. These multimodal models are accessible through OpenAI’s API, but are not currently integrated into the ChatGPT interface.

A key feature of these models is their expanded 1-million-token context window. This allows them to process approximately 750,000 words in a single input, exceeding the length of renowned works like “War and Peace.”

Competition in Advanced Programming Models

The arrival of GPT-4.1 coincides with increased efforts from competitors like Google and Anthropic to develop sophisticated models for programming. Google’s Gemini 2.5 Pro, also boasting a 1-million-token context window, performs strongly on established coding benchmarks.

Similarly, Anthropic’s Claude 3.7 Sonnet and the upgraded V3 model from Chinese AI firm DeepSeek have also achieved high rankings in coding assessments.

The Pursuit of AI-Powered Software Engineering

Many technology companies, including OpenAI, are focused on creating AI models capable of handling complex software engineering duties. OpenAI’s objective, as articulated by CFO Sarah Friar, is to develop an “agentic software engineer.”

The company envisions future models that can autonomously program complete applications, encompassing quality assurance, debugging, and documentation creation.

GPT-4.1: A Step Towards Full Automation

GPT-4.1 represents a progression toward this goal. OpenAI has refined the model based on developer feedback, concentrating on areas such as frontend coding, minimizing unnecessary edits, and ensuring consistent adherence to formatting and structure.

According to an OpenAI spokesperson, these enhancements empower developers to construct agents that are considerably more effective in real-world software engineering applications.

Performance and Efficiency Trade-offs

OpenAI asserts that the full GPT-4.1 model surpasses both GPT-4o and GPT-4o mini in coding benchmark performance, specifically on the SWE-bench test. GPT-4.1 mini and nano prioritize efficiency and speed, albeit with a slight reduction in accuracy.

Notably, OpenAI identifies GPT-4.1 nano as its fastest and most cost-effective model to date.

Pricing Details for the GPT-4.1 Models

The cost structure for the GPT-4.1 models is as follows:

• GPT-4.1: $2 per million input tokens and $8 per million output tokens
• GPT-4.1 mini: $0.40/million input tokens and $1.60/million output tokens
• GPT-4.1 nano: $0.10/million input tokens and $0.40/million output tokens

Benchmark Results and Comparisons

Internal testing by OpenAI indicates that GPT-4.1, with its capacity to process 32,768 tokens simultaneously, achieved scores between 52% and 54.6% on SWE-bench Verified. This is a human-validated subset of the SWE-bench benchmark.

However, these scores are slightly lower than those reported by Google (63.8% for Gemini 2.5 Pro) and Anthropic (62.3% for Claude 3.7 Sonnet) on the same benchmark.

Video Understanding Capabilities

In a separate evaluation utilizing Video-MME, designed to assess a model’s ability to interpret video content, GPT-4.1 achieved a leading accuracy of 72% in the “long, no subtitles” video category, according to OpenAI’s claims.

Limitations and Considerations

Despite strong benchmark performance and an updated knowledge cutoff extending to June 2024, GPT-4.1, like other advanced models, can struggle with tasks easily handled by human experts.

Studies have demonstrated that code-generating models can inadvertently introduce security vulnerabilities and bugs, or fail to correct existing ones.

Reliability and Prompt Specificity

OpenAI acknowledges that GPT-4.1’s reliability decreases as the number of input tokens increases. Accuracy dropped from approximately 84% with 8,000 tokens to 50% with 1 million tokens in company testing.

Furthermore, the company notes that GPT-4.1 tends to be more “literal” in its responses than GPT-4o, often requiring more precise and detailed prompts.