Model reference · open weights
multi-talker-whisper-small-ami is an open-weight audio or speech model from espnet. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | espnet |
|---|---|
| Type | Audio & music |
| Task | Speech→text |
| Runs with | espnet |
| Based on | openai/whisper-small |
| Released | 2026-05-24 |
| Popularity | 21 downloads / month |
| Licence | Open weights |
About
ESPnet checkpoint for serialized-output-training (SOT) multi-talker ASR on
the AMI meeting corpus, built on top of openai/whisper-small. Trained to
emit a single transcript containing every speaker in FIFO order, separated
by a `` speaker-change token, with per-speaker Whisper-style timestamps.
| File | Purpose |
|---|---|
model.pth | ESPnet-format weights (479 keys) |
config.yaml | Model architecture / preprocessor spec |
token_list.txt | 51,865-token Whisper multilingual vocabulary |
This checkpoint is consumed by the
egs2/ami/sot_asr1
recipe. After cloning ESPnet and preparing the AMI test data, place the
three files under exp/whisper-sot-small-ami/ and run:
./run.sh --released_model exp/whisper-sot-small-ami \
--whisper_model small \
--decode_test_sets test
The recipe prints utterance-group cpWER and utterance-group DER under
exp/whisper-sot-small-ami/decode_released/test/eval/.
| overall | 1-spk | 2-spk | 3-spk | 4-spk |
|---|---|---|---|---|
| 27.95 | 15.36 | 25.54 | 38.94 | 52.44 |
| overall | 1-spk | 2-spk | 3-spk | 4-spk |
|---|---|---|---|---|
| 9.84 | 1.47 | 6.99 | 18.65 | 29.43 |
From the published model card. Full card on the HuggingFace links in the sidebar.
How it works
Using it via the API
Once AxForge deploys multi-talker-whisper-small-ami for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (multi-talker-whisper-small-ami below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="multi-talker-whisper-small-ami" -F file=@audio.mp3
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.