Stability AI, the startup behind AI-based art generator Stable Diffusion, has launched an open AI model for generating sounds and songs that it claims has been trained solely on royalty-free recordings.
The generative model, called Stable Audio Open, takes a text description (e.g. “Rock beat playing in a processed studio, session drumming on an acoustic kit”) and outputs a recording up to 47 seconds long. The model was trained using approximately 486,000 samples from free music libraries FreeSound and Free Music Archive.
Stability AI can use this model to generate drum beats, instrumental riffs, ambient noises, and “production elements” for videos, movies, and TV shows, as well as to “edit” existing songs or apply a style to a song. Tell others that you can use (e.g. smooth jazz).
“The key benefit of this open source release is that users can fine-tune the model based on their own custom audio data,” Stability AI wrote in a post on the company’s blog. For example, a drummer can fine-tune samples of his drum recordings to create new beats.
However, Stable Audio Open has limitations. It cannot produce complete songs, melodies, or vocals. At least it's not a good song. Stability AI is not optimized for this, and we suggest users who want these features opt for the company's premium Stable Audio service.
Stable Audio Open is also not available for commercial use. Our Terms of Service prohibit this. And it doesn't do equally well with descriptions of musical styles and cultures or in languages other than English. Bias Stability AI blames its training data.
“Data sources likely lack diversity and not all cultures are equally represented in the data set,” Stability AI wrote in its description of the model. “The samples generated by the model reflect biases in the training data.”
Stability AI, which has long struggled to turn around its slumping business, recently resigned after Ed Newton-Rex, its vice president of generative audio, disagreed with the company's position that it was “fair” to train generative AI models on copyrighted works. Afterwards, it became the subject of controversy. Use.” Stable Audio Open appears to be an attempt to change that narrative while also not-so-subtly advertising Stability AI's paid products.
