Kimi-Audio

About

What Kimi-Audio is

Kimi-Audio Introduction We present Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation. This repository hosts the model checkpoints for Kimi-Audio-7B-Instruct. Kimi-Audio is designed as a universal audio foundation model capable of handling a wide variety of audio processing tasks within a single unified framework. Key features include: Universal Capabilities: Handles diverse tasks like speech recognition (ASR), audio question answering (AQA), audio captioning (AAC), speech emotion recognition (SER), sound event/scene classification (SEC/ASC) and end-to-end speech conversation. State-of-the-Art Performance: Achieves SOTA results on numerous audio benchmarks (see our Technical Report). Large-Scale Pre-training: Pre-trained on over 13 million hours of diverse audio data (speech, music, sounds) and text data. Novel Architecture: Employs a hybrid audio input (continuous acoustic + discrete semantic tokens) and an LLM core with parallel heads for text and audio token generation. Efficient Inference: Features a chunk-wise streaming detokenizer based on flow matching for low-latency audio generation. For more details, please refer to our GitHub Repository and Technical Report. Requirements We recommend that you build a Docker image to run the inference. After cloning the inference code, you can construct the image using the docker build command. Alternatively, You can also use our pre-built image: Or, you can install requirments by: You may refer to the Dockerfile in case of any environment issues. Quickstart This example demonstrates basic usage for generating text from audio (ASR) and generating both text and speech in a conversational turn using the Kimi-Audio-7B-Instruct model. Citation If you find Kimi-Audio useful in your research or applications, please cite our technical report: License The model is based and modified from Qwen 2.5-7B. Code derived from Qwen2.5-7B is licensed under the Apache 2.0 License. Other parts of the code are licensed under the MIT License.

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How audio & music work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys kimi-audio for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (kimi-audio below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -F model="kimi-audio" -F file=@audio.mp3

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More audio & music