Model reference · open weights
mms-tts-ara is an open-weight audio or speech model from facebook, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Massively Multilingual Speech (MMS): Arabic Text-to-Speech This repository contains the Arabic (ara) language text-to-speech (TTS) model checkpoint. This model is part of Facebook's Massively Multilingual Speech project, aiming to provide speech technology across a diverse range of languages. You can find more details about the supported languages and their ISO 639-3 codes in the MMS Language Coverage Overview, and see all MMS-TTS checkpoints on the Hugging Face Hub: facebook/mms-tts. MMS-TTS is available in the 🤗 Transformers library from version 4.33 onwards. Model Details VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior. A set of spectrogram-based acoustic features are predicted by the flow-based module, which is formed of a Transformer-based text encoder and multiple coupling layers. The spectrogram is decoded using a stack of transposed convolutional layers, much in the same style as the HiFi-GAN vocoder. Motivated by the one-to-many nature of the TTS problem, where the same text input can be spoken in multiple ways, the model also includes a stochastic duration predictor, which allows the model to synthesise speech with different rhythms from the same input text. The model is trained end-to-end with a combination of losses derived from variational lower bound and adversarial training. To improve the expressiveness of the model, normalizing flows are applied to the conditional prior distribution. During inference, the text encodings are up-sampled based on the duration prediction module, and then mapped into the waveform using a cascade of the flow module and HiFi-GAN decoder. Due to the stochastic nature of the duration predictor, the model is non-deterministic, and thus requires a fixed seed to generate the same speech waveform. For the MMS project, a separate VITS checkpoint is trained on each langauge. Usage MMS-TTS is available in the 🤗 Transformers library from version 4.33 onwards. To use this
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | |
|---|---|
| Type | Audio & music |
| Parameters (lead) | 36M |
| Variants | 1 |
| Runs with | transformers |
| Released | 2023-09-01 |
| Popularity | 65k downloads / month |
| Likes | 21 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| mms-tts-ara | 36M | BF16 | ~0.1 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys mms-tts-ara for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mms-tts-ara below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="mms-tts-ara" -F file=@audio.mp3
Licence
The weights are open but cc-by-nc-4.0 needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗