Model reference · open weights
wav2vec2-xls-r-mixed is an open-weight audio or speech model from mesolitica. wav2vec2-xls-r-300m-mixed (BF16) weighs 1.3 GB; the smallest configuration that runs it is RTX 3060 12 GB.
Summary of the mesolitica/wav2vec2-xls-r-300m-mixed model card, 2026-10-01
What it is
| Released by | mesolitica |
|---|---|
| Released | 2022-06-01 |
| VRAM | 1.3 GB for the weights |
What it runs on
| Card | One stream | Memory |
|---|---|---|
| RTX 3060 12 GB | fits | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
From the model card
Finetuned https://huggingface.co/facebook/wav2vec2-xls-r-300m on https://github.com/huseinzol05/malaya-speech/tree/master/data/mixed-stt
This model was finetuned on 3 languages,
This model trained on a single RTX 3090 Ti 24GB VRAM, provided by https://mesolitica.com/.
Evaluation set from https://github.com/huseinzol05/malaya-speech/tree/master/pretrained-model/prepare-stt with sizes,
len(malay), len(singlish), len(mandarin)
-> (765, 3579, 614)
It achieves the following results on the evaluation set based on evaluate-gpu.ipynb:
Mixed evaluation,
CER: 0.0481054244857041
WER: 0.1322198446007387
CER with LM: 0.041196586938584696
WER with LM: 0.09880169127621556
Malay evaluation,
CER: 0.051636391937588406
WER: 0.19561999547293663
CER with LM: 0.03917689630621449
WER with LM: 0.12710746406824835
Singlish evaluation,
CER: 0.0494915200071987
WER: 0.12763802881676573
CER with LM: 0.04271234986432335
WER with LM: 0.09677160640413336
Mandarin evaluation,
CER: 0.035626554824269824
WER: 0.07993515937860181
CER with LM: 0.03487760945087219
WER with LM: 0.07536807168546154
Language model from https://huggingface.co/huseinzol05/language-model-bahasa-manglish-combined
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.