Model reference · open weights
wav2vec2-large-xlsr-cantonese is an open-weight audio or speech model from scottykwok. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | scottykwok |
|---|---|
| Type | Audio & music |
| Task | Speech→text |
| Runs with | transformers |
| Released | 2022-03-02 |
| Popularity | 35k downloads / month |
| Licence | Open weights |
About
This model was based on wav2vec2-large-xlsr-53, finetuned using Common Voice/zh-HK/6.1.0.
The training code is similar to user ctl, except that the number of training epochs was 80 (doubled) and fp16_backend is apex. The model was trained using a single RTX 3090 and docker image is nvidia/cuda:11.1-cudnn8-devel.
CER is 15.11% when evaluate against common voice zh-HK test set.
15.11%
See this GitHub Repo cantonese-selfish-project and demo video.
import soundfile as sf
import torch
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
# load pretrained model
processor = Wav2Vec2Processor.from_pretrained("scottykwok/wav2vec2-large-xlsr-cantonese")
model = Wav2Vec2ForCTC.from_pretrained("scottykwok/wav2vec2-large-xlsr-cantonese")
# load audio - must be 16kHz mono
audio_input, sample_rate = sf.read('audio.wav')
# pad input values and return pt tensor
input_values = processor(audio_input, sampling_rate=sample_rate, return_tensors="pt").input_values
# INFERENCE
# retrieve logits & take argmax
logits = model(input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
# transcribe
transcription = processor.decode(predicted_ids[0])
print("-" *20)
print("Transcription:\n", transcription.lower())
print("-" *20)
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys wav2vec2-large-xlsr-cantonese for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-large-xlsr-cantonese below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-large-xlsr-cantonese" -F file=@audio.mp3
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.