Model reference · open weights
wav2vec2-lv-60-espeak-cv-ft is an open-weight audio or speech model from facebook, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Wav2Vec2-Large-LV60 finetuned on multi-lingual Common Voice This checkpoint leverages the pretrained checkpoint wav2vec2-large-lv60 and is fine-tuned on CommonVoice to recognize phonetic labels in multiple languages. When using the model make sure that your speech input is sampled at 16kHz. Note that the model outputs a string of phonetic labels. A dictionary mapping phonetic labels to words has to be used to map the phonetic output labels to output words. Paper: Simple and Effective Zero-shot Cross-lingual Phoneme Recognition Authors: Qiantong Xu, Alexei Baevski, Michael Auli Abstract Recent progress in self-training, self-supervised pretraining and unsupervised learning enabled well performing speech recognition systems without any labeled data. However, in many cases there is labeled data available for related languages which is not utilized by these methods. This paper extends previous work on zero-shot cross-lingual transfer learning by fine-tuning a multilingually pretrained wav2vec 2.0 model to transcribe unseen languages. This is done by mapping phonemes of the training languages to the target language using articulatory features. Experiments show that this simple method significantly outperforms prior work which introduced task-specific architectures and used only part of a monolingually pretrained model. The original model can be found under https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#wav2vec-20. Usage To transcribe audio files the model can be used as a standalone acoustic model as follows:
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | |
|---|---|
| Type | Audio & music |
| Variants | 1 |
| Runs with | transformers |
| Released | 2022-03-02 |
| Popularity | 179k downloads / month |
| Likes | 70 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| wav2vec2-lv-60-espeak-cv-ft | — | BF16 | — | — | Weights ↗ |
Using it via the API
Once AxForge deploys wav2vec2-lv-60-espeak-cv-ft for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-lv-60-espeak-cv-ft below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-lv-60-espeak-cv-ft" -F file=@audio.mp3
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗