Model reference · open weights
whisper-large is an open-weight audio or speech model from openai, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Whisper Whisper is a state-of-the-art model for automatic speech recognition (ASR) and speech translation, proposed in the paper Robust Speech Recognition via Large-Scale Weak Supervision by Alec Radford et al. from OpenAI. Trained on 5M hours of labeled data, Whisper demonstrates a strong ability to generalise to many datasets and domains in a zero-shot setting. Whisper large-v3-turbo is a finetuned version of a pruned Whisper large-v3. In other words, it's the exact same model, except that the number of decoding layers have reduced from 32 to 4. As a result, the model is way faster, at the expense of a minor quality degradation. You can find more details about it in this GitHub discussion. Disclaimer: Content for this model card has partly been written by the 🤗 Hugging Face team, and partly copied and pasted from the original model card. Usage Whisper large-v3-turbo is supported in Hugging Face 🤗 Transformers. To run the model, first install the Transformers library. For this example, we'll also install 🤗 Datasets to load toy audio dataset from the Hugging Face Hub, and 🤗 Accelerate to reduce the model loading time: The model can be used with the pipeline class to transcribe audios of arbitrary length: To transcribe a local audio file, simply pass the path to your audio file when you call the pipeline: Multiple audio files can be transcribed in parallel by specifying them as a list and setting the batchsize parameter: Transformers is compatible with all Whisper decoding strategies, such as temperature fallback and condition on previous tokens. The following example demonstrates how to enable these heuristics: Whisper predicts the language of the source audio automatically. If the source audio language is known a-priori, it can be passed as an argument to the pipeline: By default, Whisper performs the task of speech transcription, where the source audio language is the same as the target text language. To perform speech translation, where the target text is in English, set the task to "translate": Finally, the model can be made to predict timestamps. For sentence-level timestamps, pass the returntimestamps argument: And for word-level timestamps: The above argu
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | openai |
|---|---|
| Type | Audio & music |
| Parameters (lead) | 809M |
| Variants | 2 |
| Runs with | transformers |
| Based on | openai/whisper-large-v3 |
| Released | 2024-10-01 |
| Popularity | 7.3M downloads / month |
| Likes | 6,205 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
Benchmarks
As published on the model card — the maker's own numbers, not measured by AxForge.
| Task | Dataset | Metric | Score |
|---|---|---|---|
| Automatic Speech Recognition | LibriSpeech (clean) | Test WER | 3 |
| Automatic Speech Recognition | LibriSpeech (other) | Test WER | 5.4 |
| Automatic Speech Recognition | Common Voice 11.0 | Test WER | 54.8 |
Using it via the API
Once AxForge deploys whisper-large for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (whisper-large below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="whisper-large" -F file=@audio.mp3
Licence
Open weights under mit — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗