Model reference · open weights
wav2vec2-xls-r-juznevesti-sr is an open-weight audio or speech model from classla. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | classla |
|---|---|
| Type | Audio & music |
| Task | Speech→text |
| Runs with | transformers |
| Released | 2022-08-18 |
| Popularity | 254k downloads / month |
| Licence | Unknown |
About
This model for Serbian ASR is based on the facebook/wav2vec2-xls-r-300m model and was fine-tuned with 58 hours of audio and transcripts from Južne vesti, programme '15 minuta'.
For more info on the dataset creation see this repo.
Evaluation is performed on the dev and test portions of the JuzneVesti dataset
| dev | test | |
|---|---|---|
| WER | 0.295206 | 0.290094 |
| CER | 0.140766 | 0.137642 |
transformersTested with transformers==4.18.0, torch==1.11.0, and SoundFile==0.10.3.post1.
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import soundfile as sf
import torch
import os
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained(
"classla/wav2vec2-xls-r-juznevesti-sr")
model = Wav2Vec2ForCTC.from_pretrained("classla/wav2vec2-xls-r-juznevesti-sr")
# download the example wav files:
os.system("wget https://huggingface.co/classla/wav2vec2-xls-r-parlaspeech-hr/raw/main/00020570a.flac.wav")
# read the wav file
speech, sample_rate = sf.read("00020570a.flac.wav")
input_values = processor(speech, sampling_rate=sample_rate, return_tensors="pt").input_values.to(device)
# remove the raw wav file
os.system("rm 00020570a.flac.wav")
# retrieve logits
logits = model.to(device)(input_values).logits
# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.decode(predicted_ids[0])
transcription # 'velik broj poslovnih subjekata posluje sa minosom velik deo'
In fine-tuning, the following arguments were used:
| arg | value |
|---|---|
per_device_train_batch_size | 16 |
gradient_accumulation_steps | 4 |
num_train_epochs | 20 |
learning_rate | 3e-4 |
warmup_steps | 500 |
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys wav2vec2-xls-r-juznevesti-sr for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-xls-r-juznevesti-sr below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-xls-r-juznevesti-sr" -F file=@audio.mp3
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.