the reinforcement gap — or why some ai skills improve faster than others  

The Accelerating Divide in AI Capabilities

Artificial intelligence coding tools are experiencing rapid advancements. The extent of these changes may not be readily apparent to those outside the field of software development, but recent releases like GPT-5, Gemini 2.5, and Sonnet 4.5 have unlocked new possibilities for automating developer workflows.

Uneven Progress Across AI Domains

Conversely, progress in other areas of AI is proceeding at a slower pace. The utility gained from employing AI for tasks such as email composition remains largely unchanged over the past year. Even improvements to the underlying models don't consistently translate into tangible benefits, especially in the case of multi-purpose chatbots.

This disparity in advancement stems from a fundamental difference in how these capabilities are being improved. Reinforcement learning (RL) is now a major driver of AI progress, and its effectiveness hinges on the ability to perform automated, measurable evaluations.

The Power of Measurable Results

Coding applications are benefiting from billions of readily quantifiable tests, enabling them to generate functional code through iterative refinement. This process leverages reinforcement learning, which has become increasingly sophisticated in recent months. While human feedback can be incorporated, RL thrives when a clear pass/fail criterion exists, allowing for billions of repetitions without constant human intervention.

RL-Friendly vs. RL-Challenged Skills

As the industry increasingly relies on reinforcement learning, a clear distinction emerges between skills that can be automatically assessed and those that cannot. Capabilities like bug-fixing and competitive mathematics are improving rapidly due to their suitability for RL, while tasks like creative writing demonstrate only incremental gains.

This phenomenon creates what can be termed a “reinforcement gap” – a critical factor determining what AI systems can successfully achieve.

Software Development: An Ideal Testbed

Software development is uniquely suited for reinforcement learning. Prior to the advent of AI, a dedicated sub-discipline focused on rigorously testing software resilience, ensuring code stability before deployment. Therefore, even meticulously crafted code requires validation through unit testing, integration testing, and security testing.

Human developers routinely employ these tests to verify their code, and, as a Google director recently noted, they are equally valuable for evaluating AI-generated code. Crucially, these tests are already systematized and scalable, making them ideal for reinforcement learning applications.

The Challenge of Subjectivity

Validating subjective outputs, such as a well-written email or a compelling chatbot response, presents a significant challenge. These skills are inherently subjective and difficult to measure at scale. However, not all tasks fall neatly into these categories.

While readily available testing frameworks may not exist for complex tasks like quarterly financial reports or actuarial science, a well-funded startup could potentially develop one. The feasibility of creating such a testing framework will ultimately determine whether a process can be transformed into a viable product, rather than remaining a mere demonstration.

Surprising Testability and Future Implications

Some processes prove more testable than initially anticipated. AI-generated video, previously considered a “hard to test” area, is demonstrating rapid progress with models like OpenAI’s Sora 2. Sora 2 exhibits significant improvements in object permanence and realistic physics simulations.

It is likely that a robust reinforcement learning system underlies these advancements, focusing on qualities like consistent object representation and adherence to physical laws. These improvements collectively distinguish photorealistic video from mere visual illusions.

The Evolving Landscape of AI

It’s important to note that this is not an immutable law of artificial intelligence. The central role of reinforcement learning in AI development could shift as models evolve. However, as long as RL remains the primary method for bringing AI products to market, the reinforcement gap will likely widen, with significant consequences for both startups and the broader economy.

The ability to automate a process on the favorable side of the reinforcement gap will likely determine a startup’s success, potentially displacing workers in those fields. The question of which healthcare services are amenable to RL training, for example, has profound implications for the future economy. And, as demonstrated by surprises like Sora 2, answers may emerge sooner than expected.