VieNeu-TTS-Nano

Overview

VieNeu-TTS v3 Nano is a 48M-parameter flow-matching TTS model for Vietnamese, built for edge devices and weak CPUs: old laptops, mini PCs, ARM single-board computers and Android phones. It ships as a handful of small ONNX graphs (≈280 MB at runtime), runs torch-free on ONNX Runtime, loads in about 3 seconds and synthesizes at roughly 3× the speed of v3 Turbo on the same CPU.

It is a different architecture from v3 Turbo (a non-autoregressive flow model instead of an LLM-style autoregressive backbone), which is where the speed comes from — and also where the quality gap comes from. It supports 6 built-in preset voices, inline emotion cues ([cười], [thở dài], [hắng giọng]) and outputs 24 kHz audio.

The reference implementation is the vieneu Python SDK (v3.5.0) via Vieneu(mode="v3nano").

⚖️ Nano vs. Turbo at a glance

Choose Nano when: you need speed on a weak CPU, you are deploying on Android / ARM boards, or model load time and download size matter more than fidelity. Choose Turbo when: you want the best quality, English or bilingual text, the most faithful cloning, 48 kHz output, or real-time streaming — i.e. almost every other case.

🏗️ Architecture & Credits

The VieNeu-TTS v3 Nano architecture:

• Model: a conditional flow-matching (OT-CFM) model that maps text to codec latents in a fixed number of Euler steps, then decodes them to a waveform:
  • Text encoder — ConvNeXt blocks + self-attention over the phoneme sequence, conditioned on 50 style tokens extracted from the reference voice.
  • Duration predictor — predicts the total utterance length from the text context and the speaker embedding.
  • Vector estimator — dilated ConvNeXt blocks with FiLM conditioning on (time step, speaker) and cross-attention to the text with a length-aware rotary position bias for stable alignment; run with classifier-free guidance (default cfg=3, 16 Euler steps).
  • Speaker conditioning — a 192-d x-vector from a frozen speaker encoder plus the style tokens; each preset voice is pre-packed as {speaker_emb, style} so nothing needs to be encoded at runtime.
• Training data: predominantly Vietnamese speech from the author's corpus, with very little English — which is why English and code-switched text are the weakest part of this model.
• Audio codec: VieNeu-Codec-Nano — a 24 kHz neural codec by the author; only the decoder is shipped here (the encoder is not released, hence no voice cloning).
• Phonemizer: sea-g2p — fast Vietnamese/English grapheme-to-phoneme, also by the author.

Tác giả: Phạm Nguyễn Ngọc Bảo

📦 Using the Python SDK (vieneu)

Nano needs only the minimal, torch-free install:

pip install vieneu

from vieneu import Vieneu
from time import time

# v3 Turbo is the default: Vieneu(). Nano must be requested explicitly.
tts = Vieneu(mode="v3nano")          # ONNX, CPU, torch-free, ~3 s to load

text = "Xin chào các bạn, mình là giọng đọc của VieNeu Nano. [cười] Mình nhẹ hơn nhiều so với bản Turbo, nên chạy được cả trên những máy yếu."

# 1. Default voice (Adam) — 24 kHz
start = time()
audio = tts.infer(text)
tts.save(audio, "output_nano.wav")
print(f"{len(audio) / tts.sample_rate:.1f}s of audio in {time() - start:.2f}s")

# 2. Built-in voices by name
for label, voice_id in tts.list_preset_voices():
    print(label, voice_id)
audio = tts.infer("Mình là Ái Hân nè!", voice="Ái Hân")

# 3. Emotion / non-verbal cues — EXPERIMENTAL: [cười] [thở dài] [hắng giọng]
audio = tts.infer("Nghe hay quá đi [cười]. Để mình nói tiếp [hắng giọng].", voice="Đức Trí")

# 4. Faster on very slow CPUs: 8 Euler steps + sway sampling (~2× faster, slightly rougher)
audio = tts.infer("Bản nhanh cho máy rất yếu.", voice="Xuân Tiên", steps=8, sway=-1)

Knobs: steps (Euler steps, default 16; 8 ≈ 2× faster, pair with sway=-1), cfg (classifier-free guidance, default 3.0; cfg=0 halves compute but hurts intelligibility), speed, seed, threads (ONNX Runtime threads). infer_stream() yields one finished sentence-chunk at a time; infer_batch() runs sequentially.

[!NOTE] **Voice cloning (SDK ≥ 3.5.