Stability AI Releases Smartphone Audio Generation Model

Stability AI Launches Stable Audio Open Small

Stability AI has recently unveiled Stable Audio Open Small, a new AI model designed for generating audio. The company asserts this “stereo” model is currently the fastest available, and is capable of functioning directly on smartphones.

Collaboration with Arm

The development of Stable Audio Open Small resulted from a partnership between Stability AI and Arm, a leading chipmaker. Arm’s processors are widely used in mobile devices like tablets and phones. Unlike many AI audio applications, such as Suno and Udio, this model doesn’t necessarily require cloud processing for operation.

Royalty-Free Training Data

Stability AI emphasizes that the model’s training dataset consists exclusively of tracks sourced from royalty-free audio libraries, specifically Free Music Archive and Freesound. This contrasts with the training data used by Suno and Udio, which are reported to include copyrighted material, potentially creating intellectual property concerns.

Model Specifications and Performance

Stable Audio Open Small comprises 341 million parameters, which are the internal components that dictate the model’s behavior. It is specifically optimized for Arm CPUs. The model excels at rapidly creating brief audio samples and sound effects, like drum loops and instrumental riffs.

Stability AI claims the model can generate up to 11 seconds of audio on a smartphone in under 8 seconds.

Here’s an audio sample created by Stable Audio Open Small:

And here’s another example:

Limitations of the Model

Despite its capabilities, Stable Audio Open Small has certain limitations. Currently, it only accepts prompts written in the English language. Stability AI also notes in its documentation that the model is not designed to produce realistic vocals or high-fidelity songs.

Performance can also vary depending on the musical genre, a result of the Western-focused nature of its training data.

Usage Terms and Licensing

The usage terms for Stable Audio Open Small are somewhat restricted. It is freely available for research purposes, hobbyists, and businesses with annual revenues below $1 million. However, developers and organizations exceeding $1 million in revenue are required to obtain Stability AI’s enterprise license.

Recent Developments at Stability AI

Stability AI, the company behind the popular Stable Diffusion image-generation model, secured new funding last year. Investors, including Eric Schmidt and Sean Parker, aimed to revitalize the business.

The company faced challenges following reports of financial mismanagement under its co-founder and former CEO, Emad Mostaque, which led to staff departures and a cancelled partnership with Canva.

In recent months, Stability AI has appointed a new CEO and added filmmaker James Cameron to its board of directors. Furthermore, the company has released several new image-generation models.