Model reference · open weights
OpenVoiceV2 is an open-weight audio or speech model from myshell-ai. OpenVoiceV2 (BF16) weighs 131 MB; the smallest configuration that runs it is RTX 3060 12 GB.
What it is
| Released by | myshell-ai |
|---|---|
| Type | Audio & music |
| Task | Text→speech |
| Released | 2024-04-23 |
| Popularity | 0 downloads / month |
| Weights | 131 MB (OpenVoiceV2 (BF16), file size) |
| Licence | Open weights |
What it runs on
Weights 131 MB (file size) · overhead about 1.6 GB.
| Card | One stream | Counted memory |
|---|---|---|
| RTX 3060 12 GB | fits | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
Estimates, not measurements: the weights are the build's file size. A speech model's decoder keeps a small cache for every stream it transcribes, so memory grows with the streams and beams at once. Counted memory is 92 % of what CUDA reports for the card.
From the model card
In April 2024, we release OpenVoice V2, which includes all features in V1 and has:
Better Audio Quality. OpenVoice V2 adopts a different training strategy that delivers better audio quality.
Native Multi-lingual Support. English, Spanish, French, Chinese, Japanese and Korean are natively supported in OpenVoice V2.
Free Commercial Use. Starting from April 2024, both V2 and V1 are released under MIT License. Free for commercial use.
Please see usage for detailed instructions.
The input speech audio of OpenVoice can be in Any Language. OpenVoice can clone the voice in that speech audio, and use the voice to speak in multiple languages. For quick use, we recommend you to try the already deployed services:
This section is only for developers and researchers who are familiar with Linux, Python and PyTorch. Clone this repo, and run
conda create -n openvoice python=3.9
conda activate openvoice
git clone git@github.com:myshell-ai/OpenVoice.git
cd OpenVoice
pip install -e .
No matter if you are using V1 or V2, the above installation is the same.
Download the checkpoint from here and extract it to the checkpoints folder.
1. Flexible Voice Style Control.
Please see demo_part1.ipynb for an example usage of how OpenVoice enables flexible style control over the cloned voice.
2. Cross-Lingual Voice Cloning.
Please see demo_part2.ipynb for an example for languages seen or unseen in the MSML training set.
3. Gradio Demo.. We provide a minimalist local gradio demo here. We strongly suggest the users to look into demo_part1.ipynb, demo_part2.ipynb and the QnA if they run into issues with the gradio demo. Launch a local gradio demo with python -m openvoice_app --share.
Download the checkpoint from here and extract it to the checkpoints_v2 folder.
Install MeloTTS:
pip install git+https://github.com/myshell-ai/MeloTTS.git
python -m unidic download
Demo Usage. Please see demo_part3.ipynb for example usage of OpenVoice V2. Now it natively supports English, Spanish, French, Chinese, Japanese and Korean.
This section provides the unofficial installation guides by open-source contributors in the community:
Quoted from the model card on Hugging Face — the full card is behind the Hugging Face link above.
How it works