Stable Audio
Stable Audio is Stability AI's commercial text-to-audio generation model that produces high-quality music and sound effects from natural language descriptions.
Key Highlights
Professional Audio Quality
Produces output at 44.1 kHz stereo audio quality, the professional music production standard, delivering industry-standard results
Song Generation Up to 3 Minutes
With Stable Audio 2.0, it can generate full songs with structured intro, verse, chorus and outro sections
Audio-to-Audio Transformation
Can transform existing audio to desired styles by uploading reference audio clips and perform creative remix operations
Licensed Training Data
Trained on a fully licensed music dataset from AudioSparx, providing legal clarity and confidence for commercial outputs
About
Stable Audio is Stability AI's commercial text-to-audio generation model that produces high-quality music and sound effects from natural language descriptions. Built on a latent diffusion architecture adapted for audio, Stable Audio represents a significant advancement in AI-generated audio quality, producing professional-grade 44.1 kHz stereo audio. The model's second version (v2) can generate uninterrupted music segments up to 180 seconds in length.
Stable Audio's technical architecture consists of three core components: a variational autoencoder (VAE) that compresses audio signals into a low-dimensional latent space, a T5-based text encoder for text conditioning, and a diffusion transformer (DiT) that performs audio generation in the latent space. This architecture represents a successful adaptation of latent diffusion models used in image generation to the audio domain. The timing conditioning feature allows users to control the duration and structure of generated audio. The model is trained on AudioSparx's extensive licensed music library, ensuring a copyright-safe data foundation.
In terms of performance, Stable Audio's 44.1 kHz stereo output surpasses MusicGen's 32 kHz mono output and the quality thresholds of many other models. With version 2, the maximum generation duration was extended to 180 seconds, far exceeding MusicGen's 30-second limitation. The model demonstrates high sensitivity to text prompts, successfully interpreting detailed descriptions involving genre, tempo, instrumentation, and mood. Advanced features such as audio-to-audio generation and style transfer are also supported.
Stable Audio finds applications across a broad spectrum including professional music production, film and advertising scoring, game sound design, podcast background music, and social media content creation. It serves as a powerful alternative particularly for creators requiring professional-quality, royalty-free music. Its stereo output and extended duration support make it suitable for integration into professional workflows.
Stable Audio is accessible as a SaaS model with free and paid tiers. Its web-based interface allows instant music generation through prompt input. API access enables integration for commercial projects. A version of the model is also available as open-source on Hugging Face for research purposes, though full commercial features require the Stability AI platform.
In the AI music generation landscape, Stable Audio stands as one of the leaders of the diffusion-based approach. Compared to MusicGen and AudioCraft's autoregressive methods, the diffusion architecture delivers smoother, more professional audio quality. When compared to full song generation platforms like Suno AI and Udio that include vocals, Stable Audio is positioned as a specialized solution for instrumental music and sound design. Its 44.1 kHz stereo output standard makes it one of the most suitable options for professional audio production workflows.
Looking more closely at Stable Audio's technical features, the timing conditioning mechanism stands out as one of the model's most innovative aspects. This mechanism allows users to specify start and end times for the generated audio, enabling the creation of music pieces that fit specific durations. The multi-head attention layers in the diffusion transformer architecture are optimized to ensure long-duration musical coherence. Through the AudioSparx partnership, the model was trained on hundreds of thousands of professionally produced music tracks, and this data quality is one of the primary sources of the professional audio quality in its outputs. Stability AI has added features such as stereo rendering, improved prompt understanding, and longer generation duration through version updates. The Stable Audio Open version provides researchers with an opportunity to examine the model architecture and training approach in detail.
Use Cases
Commercial Media Production
Generate licensed high-quality music for advertising, corporate video and media projects
Podcast Background Music
Create intro, outro and transition music for podcast episodes
Game Sound Design
Generate ambient sounds, music tracks and sound effects for game environments
Film and Video Scoring
Create cinematic music and atmospheric scores for short films, documentaries and video projects
Pros & Cons
Pros
- Stability AI's text-to-audio model — music and sound effect generation
- High-quality audio output with latent diffusion architecture
- Stereo output and 44.1 kHz sampling rate
- Audio generation support up to 3 minutes
Cons
- Vocal and lyrics generation not supported
- Very limited free plan — 20 generations/month
- Quality inconsistencies in some music genres
- Stability AI's financial uncertainty
Technical Details
Parameters
N/A
Architecture
Latent diffusion model with variational autoencoder
Training Data
Proprietary licensed audio dataset from various sources
License
Stability AI Community
Features
- 44.1 kHz Stereo Output
- Up to 3-Minute Song Generation
- Audio-to-Audio Transformation
- Diffusion Transformer Architecture
- Web Interface and API Access
- Licensed Training Data (AudioSparx)
Benchmark Results
| Metric | Value | Compared To | Source |
|---|---|---|---|
| Maksimum Süre | 180 saniye (v2) | MusicGen: 30 saniye | Stability AI Blog |
| Örnekleme Hızı | 44.1 kHz | MusicGen: 32 kHz | arXiv 2407.14358 |
| FAD (MusicCaps) | 2.50 (v2) | MusicGen-Large: 3.80 | Stability AI Research |
| Parametre Sayısı | 1.1B | — | arXiv 2407.14358 |
Available Platforms
Frequently Asked Questions
Related Models
AudioCraft
AudioCraft is Meta AI's comprehensive open-source framework for generative audio research and applications, bringing together three specialized models under a single integrated platform: MusicGen for music generation, AudioGen for sound effect synthesis, and EnCodec for neural audio compression. Released in August 2023 under the MIT license, AudioCraft provides a unified codebase that simplifies working with state-of-the-art audio generation models through consistent APIs and shared infrastructure. The framework is built on a transformer-based architecture where audio signals are first compressed into discrete tokens by EnCodec, then generated autoregressively by task-specific language models. MusicGen handles text-to-music generation with melody conditioning support, while AudioGen specializes in environmental sounds, sound effects, and non-musical audio from text descriptions. EnCodec serves as the neural audio codec backbone, compressing audio at various bitrates while maintaining high perceptual quality. AudioCraft supports multiple model sizes, stereo generation, and provides extensive training and inference utilities. The framework includes pre-trained models for immediate use and tools for training custom models on user-provided datasets. As a Python library installable via pip, AudioCraft integrates seamlessly into existing machine learning and audio processing pipelines. It is widely used by researchers studying audio generation, developers building creative audio tools, content creators needing original music and sound effects, and game studios requiring dynamic audio systems. AudioCraft represents Meta's most significant contribution to open-source audio AI and has become the foundation for numerous community projects and commercial applications in the rapidly growing AI audio generation space.
AudioLDM 2
AudioLDM 2 is a unified audio generation framework developed by researchers at the Chinese University of Hong Kong and the University of Surrey, capable of producing music, sound effects, and speech from text descriptions within a single model. Building on the original AudioLDM, version 2 introduces a universal audio representation called Language of Audio that bridges the gap between different audio types by encoding them into a shared semantic space. The model combines a GPT-2 language model for understanding text inputs with an AudioMAE encoder for audio conditioning, feeding into a latent diffusion model that generates audio spectrograms which are converted to waveforms. This architecture enables AudioLDM 2 to handle diverse audio generation tasks without requiring separate specialized models for each audio type. The model achieves competitive performance across multiple benchmarks including text-to-music, text-to-sound-effects, and text-to-speech evaluations. AudioLDM 2 generates audio at up to 48 kHz with good perceptual quality for both musical and non-musical content. Released in August 2023 under a research license, the model is open source with code and pre-trained weights available on GitHub and Hugging Face. AudioLDM 2 supports audio inpainting, style transfer, and super-resolution in addition to text-conditioned generation. The model is particularly relevant for researchers studying unified audio generation, content creators needing diverse audio types from a single tool, and developers building comprehensive audio generation systems. Its unified approach to handling speech, music, and environmental sounds makes it a versatile foundation for multi-purpose audio applications.
Bark
Bark is a transformer-based text-to-audio generation model developed by Suno AI that converts text into natural-sounding speech, music, and sound effects. Released as open source under the MIT license in April 2023, Bark goes far beyond traditional text-to-speech systems by generating not only spoken words but also laughter, sighs, music, and ambient sounds from text descriptions. The model uses a GPT-style autoregressive transformer architecture with EnCodec audio tokenizer to generate audio tokens that are then decoded into waveforms. Bark supports multiple languages including English, Chinese, French, German, Hindi, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, and Turkish, making it one of the most multilingual open-source audio generation models available. The model can clone voice characteristics from short audio samples, allowing users to generate speech in specific voices or speaking styles. Bark operates in a zero-shot manner, meaning it can produce diverse outputs without task-specific fine-tuning. Generation includes natural prosody, emotion, and intonation that closely mimics human speech patterns. The model generates audio at 24 kHz sample rate with reasonable quality for most applications. As a fully open-source project with pre-trained weights available on Hugging Face and GitHub, Bark is widely used by developers building voice applications, content creators producing multilingual audio, and researchers exploring generative audio models. The model is particularly valued for its versatility in handling diverse audio types within a single unified architecture and its accessibility for rapid prototyping of audio generation applications.
MusicGen
MusicGen is a single-stage transformer-based music generation model developed by Meta AI Research as part of the AudioCraft framework. Released in June 2023 under the MIT license, MusicGen uses a single autoregressive language model operating over compressed discrete audio representations from EnCodec, unlike cascading approaches that require multiple models. The model comes in multiple sizes ranging from 300M to 3.3B parameters, allowing users to balance quality against computational requirements. MusicGen generates high-quality mono and stereo music at 32 kHz from text descriptions, supporting a wide range of genres, instruments, moods, and musical styles. Users can describe desired music using natural language prompts specifying genre, tempo, instrumentation, and atmosphere, and the model produces coherent musical compositions that follow the specified characteristics. Beyond text-to-music generation, MusicGen supports melody conditioning where an existing audio clip guides the melodic structure of the generated output, enabling more controlled music creation. The model achieves strong results across both objective metrics and subjective listening evaluations, producing music that sounds natural and musically coherent for durations up to 30 seconds. As a fully open-source model with code and weights available on GitHub and Hugging Face, MusicGen has become one of the most widely adopted AI music generation tools in both research and creative communities. It integrates easily into existing audio production workflows through the Audiocraft Python library and various community-built interfaces. MusicGen is particularly popular among content creators, game developers, and musicians who need royalty-free background music generated on demand.
Quick Info
Latent diffusion model with variational autoencoder