Model reference · open weights
whisper_tflite_models is an open-weight audio or speech model from DocWolle. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | DocWolle |
|---|---|
| Type | Audio & music |
| Task | Speech→text |
| Released | 2024-12-27 |
| Popularity | 17k downloads / month |
| Licence | Open weights |
About
"transcribe-translate" models provide signatures for "serving_transcribe" and "serving_translate" to force the model to perform a certain action
@tf.function(
input_signature=[
tf.TensorSpec((1, 80, 3000), tf.float32, name="input_features"),
],
)
def transcribe(self, input_features):
outputs = self.model.generate(
input_features,
max_new_tokens=450, # change as needed
return_dict_in_generate=True,
forced_decoder_ids=[[2, 50359], [3, 50363]], # forced to transcribe any language with no timestamps
)
return {"sequences": outputs["sequences"]}
@tf.function(
input_signature=[
tf.TensorSpec((1, 80, 3000), tf.float32, name="input_features"),
],
)
def translate(self, input_features):
outputs = self.model.generate(
input_features,
max_new_tokens=450, # change as needed
return_dict_in_generate=True,
forced_decoder_ids=[[2, 50358], [3, 50363]], # forced to translate any language with no timestamps
)
return {"sequences": outputs["sequences"]}
In order to force transcription for a certain language set the 1. decoder id as shown below:
def transcribe(self, input_features):
outputs = self.model.generate(
input_features,
max_new_tokens=450, # change as needed
return_dict_in_generate=True,
forced_decoder_ids=[[1, 50261], [2, 50359], [3, 50363]], # forced to transcribe (50359) German (50261) with no timestamps (50363)
)
return {"sequences": outputs["sequences"]}
def translate(self, input_features):
outputs = self.model.generate(
input_features,
max_new_tokens=450, # change as needed
return_dict_in_generate=True,
forced_decoder_ids=[[1, 50261], [2, 50358], [3, 50363]], # different forced_decoder_ids
)
return {"sequences": outputs["sequences"]}
(language codes from here: https://github.com/woheller69/whisperIME/blob/master/app/src/main/java/com/whispertflite/utils/InputLang.java)
The models are based on:
@misc{radford2022whisper,
doi = {10.48550/ARXIV.2212.04356},
url = {https://arxiv.org/abs/2212.04356},
author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
title = {Robust Speech Recognition via Large-Scale Weak Supervision},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}
Conversion to tflite is based on:
@misc{nyadla-sys,
author={Niranjan Yadla},
title={{Whisper TFLite: OpenAI Whisper Model Port for Edge Devices}},
year=2022,
howpublished={GitHub Repository},
url={https://github.com/nyadla-sys/whisper.tflite},
note={Original TFLite implementation of OpenAI Whisper for on-device automatic speech recognition}
}
From the published model card. Full card on the HuggingFace links in the sidebar.
How it works
Using it via the API
Once AxForge deploys whisper-tflite-models for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (whisper-tflite-models below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="whisper-tflite-models" -F file=@audio.mp3
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.