A.X-K2-Raon-Speech

A.X K2 Raon-Speech is a bilingual English/Korean speech language model with approximately 21.2B total parameters and 3.5B active parameters. Built on SK Telecom's A.X K2 Light 20B-A3B mixture-of-experts text backbone, it integrates an AuT speech encoder and a Mimi-style neural audio codec independently trained by KRAFTON AI, unifying speech understanding and generation in a single multimodal model. It is trained with the recipe of Raon-Speech-9B, KRAFTON's open-source speech language model.

Key Features

• End-to-End Speech Language Model: 21.2B-parameter (approximately 3.5B active) multimodal model built on SK Telecom's A.X K2 Light 20B-A3B mixture-of-experts backbone.
• Independently Trained Speech Modules: AuT speech encoder (approximately 317M parameters) and Mimi-style neural audio codec (approximately 96M parameters), trained on curated data with an in-house pipeline.
• Bilingual Support: First in Korean and third in English aggregate score among publicly available speech language models at or below the 30B scale.
• Multi-Task Capabilities: STT, TTS, SpeechQA, SpokenQA, and turn-based multimodal chat in a single unified model.
• Paralinguistic Awareness: Uses emotion, intonation, and other information carried in the user's voice to generate natural spoken responses.
• Speaker Voice Conditioning: TTS with optional speaker reference audio via ECAPA embeddings.
• TTS Continuation: Generates speech that naturally continues from a reference audio with prefill-based continuation for seamless prosody.

Architecture

All components for the core speech-understanding and direct-TTS paths are included in the checkpoint. Speaker-conditioned and continuation TTS additionally use a frozen SpeechBrain ECAPA speaker encoder, downloaded automatically on first use.

Evaluation

Evaluation covered six task areas — speech recognition, speech synthesis, speech understanding, spoken question answering, text question answering, and tool calling — using 46 benchmarks (24 Korean, 22 English), including LibriSpeech, KsponSpeech, VoiceBench, KVoiceBench, MMAU, KMMAU, API-Bank, and FunctionChat-Bench. Each result is normalized to the 0–1 range and aggregated per language; see the KRAFTON AI tech blog for the full methodology and results.

Among publicly available speech language models at or below the 30B scale, A.X K2 Raon-Speech ranked first on the Korean aggregate score (0.72) and third on the English aggregate score (0.75).

Per-category performance, normalized from zero to the highest score in each category. Comparison models are the top English performers below that support both English and Korean.

Aggregate scores

Speech encoder

The AuT speech encoder matches Qwen3-ASR AuT on SpeechLM subtasks while trained on approximately 1M hours of audio versus the 40M hours reported for Qwen3-ASR-1.7B AuT (values relative to Qwen AuT = 100%).

Speech codec

The codec approaches Mimi's performance (values relative to Mimi = 100%) while trained on approximately 500K hours of audio; Moshi reports a 7M-hour unsupervised corpus for audio pretraining.

Post-training ablations

Against the immediately preceding checkpoint, tool-use fine-tuning improved the tool-call score from 0.543 to 0.740, paralinguistic self-distillation improved the emotion/speaker score from 0.566 to 0.613, and TTS direct preference optimization reduced the average TTS error rate from 3.6% to 3.1%.

Requirements

pip install -r requirements.txt

# Optional: lower-latency attention on supported CUDA systems
pip install flash-attn --no-build-isolation

Requires transformers>=4.57.1. speechbrain is required for the speaker-conditioned and continuation TTS APIs.

Load the model

from transformers import AutoConfig
from transformers.dynamic_module_utils