Model reference · open weights
VibeVoice is an open-weight audio or speech model from microsoft, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
VibeVoice: A Frontier Open-Source Text-to-Speech Model VibeVoice is a novel framework designed for generating expressive, long-form, multi-speaker conversational audio, such as podcasts, from text. It addresses significant challenges in traditional Text-to-Speech (TTS) systems, particularly in scalability, speaker consistency, and natural turn-taking. A core innovation of VibeVoice is its use of continuous speech tokenizers (Acoustic and Semantic) operating at an ultra-low frame rate of 7.5 Hz. These tokenizers efficiently preserve audio fidelity while significantly boosting computational efficiency for processing long sequences. VibeVoice employs a next-token diffusion framework, leveraging a Large Language Model (LLM) to understand textual context and dialogue flow, and a diffusion head to generate high-fidelity acoustic details. The model can synthesize speech up to 90 minutes long with up to 4 distinct speakers, surpassing the typical 1-2 speaker limits of many prior models. ➡️ Technical Report: VibeVoice Technical Report ➡️ Project Page: microsoft/VibeVoice ➡️ Code: microsoft/VibeVoice-Code Training Details Transformer-based Large Language Model (LLM) integrated with specialized acoustic and semantic tokenizers and a diffusion-based decoding head. - LLM: Qwen2.5-1.5B for this release. - Tokenizers: - Acoustic Tokenizer: Based on a σ-VAE variant (proposed in LatentLM), with a mirror-symmetric encoder-decoder structure featuring 7 stages of modified Transformer blocks. Achieves 3200x downsampling from 24kHz input. Encoder/decoder components are ~340M parameters each. - Semantic Tokenizer: Encoder mirrors the Acoustic Tokenizer's architecture (without VAE components). Trained with an ASR proxy task. - Diffusion Head: Lightweight module (4 layers, ~123M parameters) conditioned on LLM hidden states. Predicts acoustic VAE features using a Denoising Diffusion Probabilistic Models (DDPM) process. Uses Classifier-Free Guidance (CFG) and DPM-Solver (and variants) during inference. - Context Length: Trained with a curriculum increasing up to 65,536 tokens. - Training Stages: - Tokenizer Pre-training: Acoustic and Semantic tokenizers are pre-trained separately. - VibeV
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | microsoft |
|---|---|
| Type | Audio & music |
| Parameters (lead) | 2.7B |
| Variants | 1 |
| Runs with | transformers |
| Released | 2025-08-25 |
| Popularity | 141k downloads / month |
| Likes | 2,468 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| VibeVoice-1.5B | 2.7B | BF16 | ~6.2 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys vibevoice for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (vibevoice below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="vibevoice" -F file=@audio.mp3
Licence
Open weights under mit — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗