Model reference · open weights
wav2vec2-960h is an open-weight audio or speech model from facebook, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Wav2Vec2-Base-960h Facebook's Wav2Vec2 The base model pretrained and fine-tuned on 960 hours of Librispeech on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. Paper Authors: Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli Abstract We show for the first time that learning powerful representations from speech audio alone followed by fine-tuning on transcribed speech can outperform the best semi-supervised methods while being conceptually simpler. wav2vec 2.0 masks the speech input in the latent space and solves a contrastive task defined over a quantization of the latent representations which are jointly learned. Experiments using all labeled data of Librispeech achieve 1.8/3.3 WER on the clean/other test sets. When lowering the amount of labeled data to one hour, wav2vec 2.0 outperforms the previous state of the art on the 100 hour subset while using 100 times less labeled data. Using just ten minutes of labeled data and pre-training on 53k hours of unlabeled data still achieves 4.8/8.2 WER. This demonstrates the feasibility of speech recognition with limited amounts of labeled data. The original model can be found under https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#wav2vec-20. Usage To transcribe audio files the model can be used as a standalone acoustic model as follows: Evaluation This code snippet shows how to evaluate facebook/wav2vec2-base-960h on LibriSpeech's "clean" and "other" test data. Result (WER):
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | |
|---|---|
| Type | Audio & music |
| Parameters (lead) | 94M |
| Variants | 1 |
| Runs with | transformers |
| Released | 2022-03-02 |
| Popularity | 1.4M downloads / month |
| Likes | 402 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| wav2vec2-base-960h | 94M | BF16 | ~0.2 GB | ✓ | Weights ↗ |
Benchmarks
As published on the model card — the maker's own numbers, not measured by AxForge.
| Task | Dataset | Metric | Score |
|---|---|---|---|
| Automatic Speech Recognition | LibriSpeech (clean) | Test WER | 3.4 |
| Automatic Speech Recognition | LibriSpeech (other) | Test WER | 8.6 |
Using it via the API
Once AxForge deploys wav2vec2-960h for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wav2vec2-960h below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="wav2vec2-960h" -F file=@audio.mp3
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗