T-one

Table of Contents

1. Project Summary
2. Quality benchmarks
3. Inference examples
4. Fine-tuning
5. Acoustic model
6. Training details
7. License

📝 Project Summary

Key Features:

• Streaming-first Architecture: Built for low-latency, real-time applications.
• Ready-to-Use Pipeline: Includes a pretrained acoustic model, phrase splitter, and a KenLM-based CTC beam search decoder with examples for offline and streaming speech recognition inference.
• Demo — launch a local speech recognition service instantly via Docker and transcribe audio files or real-time microphone input.
• Fine-tuning T-one on a custom dataset is straightforward using the 🤗 ecosystem.
• Easy Deployment: Includes examples for deploying with Triton Inference Server for high-throughput scenarios.
• Fully Open Source architecture: All model and pipeline code is available.

📊 Quality benchmarks

Word Error Rate (WER) is used to evaluate the quality of automatic speech recognition systems, which can be interpreted as the percentage of incorrectly recognized words compared to a reference transcript. A lower value indicates higher accuracy. T-one demonstrates state-of-the-art performance, especially on its target domain of telephony, while remaining competitive on general-purpose benchmarks.

👨‍💻 Inference examples

Offline Inference (for entire audio files)

from tone import StreamingCTCPipeline, read_audio, read_example_audio

audio = read_example_audio() # or read_audio("your_audio.flac")

pipeline = StreamingCTCPipeline.from_hugging_face()
print(pipeline.forward_offline(audio))  # run offline recognition

Output:

[TextPhrase(text='привет', start_time=1.79, end_time=2.04), TextPhrase(text='это я', start_time=3.72, end_time=4.26), TextPhrase(text='я подумала не хочешь ли ты встретиться спустя все эти годы', start_time=5.88, end_time=10.59)]

Streaming Inference (for real-time audio)

from tone import StreamingCTCPipeline, read_stream_example_audio

pipeline = StreamingCTCPipeline.from_hugging_face()

state = None  # Current state of the ASR pipeline (None - initial)
for audio_chunk in read_stream_example_audio():  # Use any source of audio chunks
    new_phrases, state = pipeline.forward(audio_chunk, state)
    print(new_phrases)

# Finalize the pipeline and get the remaining phrases
new_phrases, _ = pipeline.finalize(state)
print(new_phrases)

Output:

TextPhrase(text='привет', start_time=1.79, end_time=2.04)
TextPhrase(text='это я', start_time=3.72, end_time=4.26)
TextPhrase(text='я подумала не хочешь ли ты встретиться спустя все эти годы', start_time=5.88, end_time=10.59)

🔧 Fine-tuning

In order to fine-tune T-one from a pre-trained checkpoint you need to prepare the training dataset, load the tokenizer and feature extractor from t-tech/T-one 🤗 repo.

import torch

from tone.training.model_wrapper import ToneForCTC

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = ToneForCTC.from_pretrained("t-tech/T-one").to(device)

Setup the data collator, evaluation metric, training arguments and 🤗 Trainer.

For a complete guide please refer to the fine-tuning example notebook.

🎙 Acoustic model

Architecture

T-one is a 71M parameter acoustic model based on the Conformer architecture, with several key innovations to improve performance and efficiency:

• SwiGLU Activation: The feed-forward module is replaced with a SwiGLU module for better performance.
• Modern Normalization: SiLU (Swish) activations and RMSNorm are used in place of ReLU and LayerNorm.
• RoPE Embeddings: Relative positional embeddings from Transformer-XL are replaced with faster Rotary Position Embeddings (RoPE).
• U-Net Structure: The temporal dimension is downsampled and then upsampled within the Conformer blocks, improving the model's receptive field.
• Attention Score Reuse: Multi-Head Self-Attention layers are grouped, and attention scores are computed only once per group to reduce computation.
• Efficient State Management: Streaming states are used only in the final two layers of the model.

It processes audio in 300 ms chunks and generates transcriptions using either greedy decoding or a KenLM-based CTC beam search de