thinking machines lab wants to make ai models more consistent

Thinking Machines Lab's Approach to Deterministic AI

Significant attention has been directed towards the advancements being made at Mira Murati’s Thinking Machines Lab, fueled by $2 billion in initial funding and the expertise of former OpenAI researchers.

On Wednesday, the lab unveiled its initial research focus through a blog post: the development of AI models capable of consistently producing the same responses to identical prompts.

Understanding AI Response Variability

The research, detailed in a post titled “Defeating Nondeterminism in LLM Inference,” investigates the origins of randomness in AI model outputs. For instance, repeated queries to ChatGPT often yield diverse answers.

This variability has generally been accepted within the AI field as an inherent characteristic of current models—considered non-deterministic systems—however, Thinking Machines Lab posits that this is a challenge that can be overcome.

The Role of GPU Kernels

Horace He, a researcher at Thinking Machines Lab and author of the post, proposes that the randomness stems from the way GPU kernels function during inference.

These kernels, which are the fundamental programs operating within Nvidia’s chips, are assembled during the processing phase following user input. He suggests that greater control over this orchestration process could lead to more deterministic AI behavior.

Benefits of Reproducible AI Responses

Achieving reproducible responses offers benefits beyond simply providing more reliable outputs for businesses and scientific applications.

It could also enhance reinforcement learning (RL) training, a method of rewarding AI models for accurate answers. Inconsistent responses introduce noise into the training data, hindering the process.

More consistent outputs could streamline RL, as previously reported by The Information, aligning with Thinking Machines Lab’s plans to utilize RL for customized AI model development for businesses.

Upcoming Product and Research Transparency

Mira Murati, formerly the CTO of OpenAI, announced in July that the lab’s first product will be released in the coming months.

This product is intended for researchers and startups focused on creating bespoke models, and may incorporate techniques aimed at generating more reproducible responses.

Thinking Machines Lab has committed to regularly publishing research findings, code, and other information to benefit both the wider community and its internal research culture.

Open Research vs. Closed Development

This initial post, part of a new series called “Connectionism,” reflects this commitment.

OpenAI initially embraced open research, but has become less transparent as it has grown. It remains to be seen whether Thinking Machines Lab will maintain its stated dedication to openness.

A Glimpse into a Secretive Startup

The research blog provides a unique insight into one of Silicon Valley’s most discreet AI startups.

While it doesn’t fully reveal the lab’s long-term technological direction, it demonstrates that Thinking Machines Lab is addressing fundamental challenges in AI research.

Ultimately, the lab’s success will depend on its ability to solve these problems and develop marketable products that justify its substantial $12 billion valuation.