Model reference · open weights
wav2vec2-xls-r-mixed is an open-weight audio or speech model from mesolitica, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
probably proofread and complete it, then remove this comment. -- wav2vec2-xls-r-300m-mixed Finetuned https://huggingface.co/facebook/wav2vec2-xls-r-300m on https://github.com/huseinzol05/malaya-speech/tree/master/data/mixed-stt This model was finetuned on 3 languages, 1. Malay 2. Singlish 3. Mandarin This model trained on a single RTX 3090 Ti 24GB VRAM, provided by https://mesolitica.com/. Evaluation set Evaluation set from https://github.com/huseinzol05/malaya-speech/tree/master/pretrained-model/prepare-stt with sizes, It achieves the following results on the evaluation set based on evaluate-gpu.ipynb: Mixed evaluation, Malay evaluation, Singlish evaluation, Mandarin evaluation, Language model from https://huggingface.co/huseinzol05/language-model-bahasa-manglish-combined
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | mesolitica |
|---|---|
| Type | Audio & music |
| Variants | 1 |
| Runs with | transformers |
| Released | 2022-06-01 |
| Popularity | 1.1M downloads / month |
| Likes | 5 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| wav2vec2-xls-r-300m-mixed | — | BF16 | — | — | Weights ↗ |
Using it via the API
Once AxForge deploys wav2vec2-xls-r-mixed for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-xls-r-mixed below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-xls-r-mixed" -F file=@audio.mp3
Details
Tags
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗