canary-qwen

img { display: inline; }

| |

Model Overview

Description:

NVIDIA NeMo Canary-Qwen-2.5B is an English speech recognition model that achieves state-of-the art performance on multiple English speech benchmarks. With 2.5 billion parameters and running at 418 RTFx, Canary-Qwen-2.5B supports automatic speech-to-text recognition (ASR) in English with punctuation and capitalization (PnC). The model works in two modes: as a transcription tool (ASR mode) and as an LLM (LLM mode). In ASR mode, the model is only capable of transcribing the speech into text, but does not retain any LLM-specific skills such as reasoning. In LLM mode, the model retains all of the original LLM capabilities, which can be used to post-process the transcript, e.g. summarize it or answer questions about it. In LLM mode, the model does not "understand" the raw audio anymore - only its transcript. This model is ready for commercial use.

License/Terms of Use:

Canary-Qwen-2.5B is released under the CC-BY-4.0 license. By using this model, you are agreeing to the terms and conditions of the license.

Discover more from NVIDIA:

For documentation, deployment guides, enterprise-ready APIs, and the latest open models—including Nemotron and other cutting-edge speech, translation, and generative AI—visit the NVIDIA Developer Portal at developer.nvidia.com. Join the community to access tools, support, and resources to accelerate your development with NVIDIA’s NeMo, Riva, NIM, and foundation models.

Explore more from NVIDIA:

What is Nemotron? NVIDIA Developer Nemotron NVIDIA Riva Speech NeMo Documentation

References:

[1] Less is More: Accurate Speech Recognition & Translation without Web-Scale Data

[2] Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition

[3] Attention Is All You Need

[4] Qwen/Qwen3-1.7B Model Card

[5] Training and Inference Efficiency of Encoder-Decoder Speech Models

[6] NVIDIA NeMo Toolkit

[7] Granary: Speech Recognition and Translation Dataset in 25 European Languages

[8] Towards Measuring Fairness in AI: the Casual Conversations Dataset

[9] SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation

Deployment Geography:

Global

Use Case:

The model is intended for users requiring speech-to-text transcription capabilities for English speech, and/or transcript post-processing capabilities enabled by prompting the underlying LLMs. Typical use-cases: transcription, summarization, answering user questions about the transcript.

Release Date:

Huggingface 07/17/2025 via https://huggingface.co/nvidia/canary-qwen-2.5b

Discover more from NVIDIA:

For documentation, deployment guides, enterprise-ready APIs, and the latest open models—including Nemotron and other cutting-edge speech, translation, and generative AI—visit the NVIDIA Developer Portal at developer.nvidia.com. Join the community to access tools, support, and resources to accelerate your development with NVIDIA’s NeMo, Riva, NIM, and foundation models.

Explore more from NVIDIA:

What is Nemotron? NVIDIA Developer Nemotron NVIDIA Riva Speech NeMo Documentation

Model Architecture:

Canary-Qwen is a Speech-Augmented Language Model (SALM) [9] model with FastConformer [2] Encoder and Transformer Decoder [3]. It is built using two base models: nvidia/canary-1b-flash [1,5] and Qwen/Qwen3-1.7B [4], a linear projection, and low-rank adaptation (LoRA) applied to the LLM. The audio encoder computes audio representation that is mapped to the LLM embedding space via a linear projection, and concatenated with the embeddings of text tokens. The model is prompted with "Transcribe the following: ", using Qwen's chat template.

Limitations

Input length. The maximum audio duration in training was 40s, and the maximum token sequence length was 1024 tokens (including prompt, audio, and response). The model may technically be able to process longer sequences, but its accuracy may be degraded.

Exclusively ASR oriented capabilities. The model is not expected to preserve any of the underlying LLM's capabilities into speech modality.

English-only language support. The model was trained using English data only. It may be able to spuriously transcribe other languages as the underlying encoder was pretrained using German, French, and Spanish speech in addition to English, but it's unlikely to be reliable as a multilingual model.

NVIDIA NeMo

To train, fine-tune or transcribe with Canary-Qwen-2.5B, you will need to install NVIDIA NeMo.

# Currently requires installing the latest trunk version of NeMo, and PyTorch 2.6+ for FSDP2 support.
python -m pip install "nemo_toolkit[asr,tts] @ git+https://gith