JetBrains Releases Mellum: An Open AI Coding Model

JetBrains Releases Open-Source AI Coding Model

JetBrains, the creator of numerous widely-used application development tools, has announced the release of its inaugural openly available AI model tailored for coding tasks.

Mellum: A Code-Generating Model

On Wednesday, JetBrains made Mellum, a model initially deployed within its suite of software development tools last year, accessible to the public via the AI development platform Hugging Face. Mellum is specifically engineered for code completion, predicting and generating code snippets based on existing context.

The model was trained on a massive dataset exceeding 4 trillion tokens and comprises 4 billion parameters. Parameters are indicative of a model’s capacity for problem-solving, while tokens represent the fundamental data units processed.

Technical Specifications and Applications

A million tokens is roughly equivalent to approximately 30,000 lines of code. JetBrains details in a technical report that Mellum is designed for diverse applications.

• Integration into professional developer tools.
• AI-powered coding assistants.
• Research into code understanding and generation.
• Educational purposes.
• Fine-tuning experiments.

The model is Apache 2.0 licensed, facilitating open use and modification.

Training and Infrastructure

JetBrains trained Mellum utilizing a compilation of datasets, including permissively licensed code sourced from GitHub and articles from the English-language Wikipedia. The training process required approximately 20 days, utilizing a cluster of 256 H200 Nvidia GPUs.

Implementation and Fine-Tuning

Implementing Mellum requires some initial effort. The base model necessitates fine-tuning before it can be utilized effectively. JetBrains has provided several pre-fine-tuned models for Python, but emphasizes these are intended for evaluating potential capabilities rather than immediate production deployment.

Security Considerations

The increasing use of AI-generated code is reshaping software development, but also introduces new security risks. A recent survey conducted in late 2023 by Snyk, a developer security platform, revealed that over 50% of organizations frequently or occasionally encounter security issues stemming from AI-produced code.

JetBrains acknowledges that Mellum may exhibit biases present in the public codebases it was trained on, potentially replicating coding styles found in open-source repositories. Furthermore, the model’s suggestions are not guaranteed to be secure or free from vulnerabilities.

Future Outlook

“This represents only the initial phase of our work,” JetBrains stated in a blog post. “Our focus isn’t on achieving broad generality, but rather on developing specialized capabilities. If Mellum inspires even a single valuable experiment, contribution, or collaboration, we will consider it a success.”