Zonos-transformer

About

What Zonos-transformer is

Zonos-v0.1 alt="Title card" style="width: 500px; height: auto; object-position: center top;" Zonos-v0.1 is a leading open-weight text-to-speech model trained on more than 200k hours of varied multilingual speech, delivering expressiveness and quality on par with—or even surpassing—top TTS providers. Our model enables highly natural speech generation from text prompts when given a speaker embedding or audio prefix, and can accurately perform speech cloning when given a reference clip spanning just a few seconds. The conditioning setup also allows for fine control over speaking rate, pitch variation, audio quality, and emotions such as happiness, fear, sadness, and anger. The model outputs speech natively at 44kHz. For more details and speech samples, check out our blog here We also have a hosted version available at playground.zyphra.com/audio Zonos follows a straightforward architecture: text normalization and phonemization via eSpeak, followed by DAC token prediction through a transformer or hybrid backbone. An overview of the architecture can be seen below. alt="Architecture diagram" style="width: 1000px; height: auto; object-position: center top;" Usage Python Gradio interface (recommended) This should produce a sample.wav file in your project root directory. For repeated sampling we highly recommend using the gradio interface instead, as the minimal example needs to load the model every time it is run. Features - Zero-shot TTS with voice cloning: Input desired text and a 10-30s speaker sample to generate high quality TTS output - Audio prefix inputs: Add text plus an audio prefix for even richer speaker matching. Audio prefixes can be used to elicit behaviours such as whispering which can otherwise be challenging to replicate when cloning from speaker embeddings - Multilingual support: Zonos-v0.1 supports English, Japanese, Chinese, French, and German - Audio quality and emotion control: Zonos offers fine-grained control of many aspects of the generated audio. These include speaking rate, pitch, maximum frequency, audio quality, and various emotions such as happiness, anger, sadness, and fear. - Fast: our model runs with a real-time factor of ~2x on an RTX 4

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How audio & music work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys zonos-transformer for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (zonos-transformer below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -F model="zonos-transformer" -F file=@audio.mp3

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More audio & music