Model reference · open weights
wav2vec2-xls-r-bengali is an open-weight audio or speech model from arijitx, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the OPENSLRSLR53 - bengali dataset. It achieves the following results on the evaluation set. Without language model : - WER: 0.21726385291857586 - CER: 0.04725010353701041 With 5 gram language model trained on 30M sentences randomly chosen from AI4Bharat IndicCorp dataset : - WER: 0.15322879016421437 - CER: 0.03413696666806267 Note : 5% of a total 10935 samples have been used for evaluation. Evaluation set has 10935 examples which was not part of training training was done on first 95% and eval was done on last 5%. Training was stopped after 180k steps. Output predictions are available under files section. Training hyperparameters The following hyperparameters were used during training: - datasetname="openslr" - modelnameorpath="facebook/wav2vec2-xls-r-300m" - datasetconfigname="SLR53" - outputdir="./wav2vec2-xls-r-300m-bengali" - overwriteoutputdir - numtrainepochs="50" - perdevicetrainbatchsize="32" - perdeviceevalbatchsize="32" - gradientaccumulationsteps="1" - learningrate="7.5e-5" - warmupsteps="2000" - lengthcolumnname="inputlength" - evaluationstrategy="steps" - textcolumnname="sentence" - charstoignore , ? . ! \- \; \: \" “ % ‘ ” � — ’ … – - savesteps="2000" - evalsteps="3000" - loggingsteps="100" - layerdrop="0.0" - activationdropout="0.1" - savetotallimit="3" - freezefeatureencoder - featprojdropout="0.0" - masktimeprob="0.75" - masktimelength="10" - maskfeatureprob="0.25" - maskfeaturelength="64" - preprocessingnumworkers 32 Framework versions - Transformers 4.16.0.dev0 - Pytorch 1.10.1+cu102 - Datasets 1.17.1.dev0 - Tokenizers 0.11.0 Notes - Training and eval code modified from : https://github.com/huggingface/transformers/tree/master/examples/researchprojects/robust-speech-event. - Bengali speech data was not available from common voice or librispeech multilingual datasets, so OpenSLR53 has been used. - Minimum audio duration of 0.5s has been used to filter the training data which excluded may be 10-20 samples. - OpenSLR53 transcripts are not part of LM training and LM used to evaluate.
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | arijitx |
|---|---|
| Type | Audio & music |
| Variants | 1 |
| Runs with | transformers |
| Released | 2022-03-02 |
| Popularity | 934k downloads / month |
| Likes | 10 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| wav2vec2-xls-r-300m-bengali | — | BF16 | — | — | Weights ↗ |
Benchmarks
As published on the model card — the maker's own numbers, not measured by AxForge.
| Task | Dataset | Metric | Score |
|---|---|---|---|
| Speech Recognition | Open SLR | Test WER | 0.217 |
| Speech Recognition | Open SLR | Test CER | 0.047 |
| Speech Recognition | Open SLR | Test WER with lm | 0.153 |
| Speech Recognition | Open SLR | Test CER with lm | 0.034 |
Using it via the API
Once AxForge deploys wav2vec2-xls-r-bengali for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-xls-r-bengali below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-xls-r-bengali" -F file=@audio.mp3
Details
Languages
Trained / evaluated on
Tags
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗