fastconformer-quran-ar

Model Details

Training

Fine-tuned using a 3-phase progressive unfreezing strategy on a single NVIDIA RTX 4070 Ti (12 GB):

Progressive unfreezing prevents catastrophic forgetting of the base model's Arabic speech representations while allowing the full model to adapt to Quranic phonetics, tajweed rules, and recitation style.

Training data: tarteel-ai/everyayah — a diverse multi-reciter dataset of complete Quranic recitations at multiple audio qualities, covering all 114 surahs across dozens of reciters.

Usage

Installation

pip install nemo_toolkit[asr]

Offline transcription (recommended for files)

The .nemo file is saved with full bilateral attention context — transcribe() works out of the box with no configuration required.

import nemo.collections.asr as nemo_asr

model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(
    "mohammed/fastconformer-quran-ar"
)
model.eval()

# Transcribe a .wav file (16kHz mono)
result = model.transcribe(["recitation.wav"])
print(result[0].text)
# e.g. "بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ"

Real-time streaming

The model supports cache-aware streaming inference via NeMo's cache_aware_stream_step(). The key loading sequence (order matters):

import torch
import nemo.collections.asr as nemo_asr
from omegaconf import OmegaConf, open_dict

model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(
    "mohammed/fastconformer-quran-ar"
)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

# Step 1 — reset conv padding to symmetric (safety check before mode switch)
for layer in model.encoder.layers:
    if hasattr(layer, "conv") and hasattr(layer.conv, "conv"):
        conv = layer.conv.conv
        ks = conv.kernel_size[0] if isinstance(conv.kernel_size, tuple) else conv.kernel_size
        conv.padding = ((ks - 1) // 2, (ks - 1) // 2)

# Step 2 — switch to causal local attention for streaming
model.change_attention_model(
    self_attention_model="rel_pos_local_attn",
    att_context_size=[128, 0],   # 128 frames lookback (~10s), fully causal
)
with open_dict(model.cfg):
    model.cfg.encoder.conv_context_size = "causal"

# Step 3 — causal conv padding
for layer in model.encoder.layers:
    if hasattr(layer, "conv") and hasattr(layer.conv, "conv"):
        conv = layer.conv.conv
        ks = conv.kernel_size[0] if isinstance(conv.kernel_size, tuple) else conv.kernel_size
        conv.padding = (ks - 1, 0)

# Step 4 — greedy decoder
decoding_cfg = OmegaConf.structured(model.cfg.decoding)
OmegaConf.set_struct(decoding_cfg, False)
decoding_cfg.strategy = "greedy"
decoding_cfg.greedy.max_symbols = 10
decoding_cfg.greedy.use_cuda_graph_decoder = False  # incompatible with streaming
model.change_decoding_strategy(decoding_cfg)
model.eval()

# Streaming loop — feed 80ms PCM int16 frames
cache_last_channel, cache_last_time = None, None
chunk_samples = 16000 * 1600 // 1000  # 1600ms chunk

audio_chunk = torch.zeros(1, chunk_samples, device=device)  # replace with real audio
audio_len   = torch.tensor([chunk_samples], device=device)

with torch.no_grad():
    processed, processed_len = model.preprocessor(
        input_signal=audio_chunk, length=audio_len
    )
    encoded, encoded_len, cache_last_channel, cache_last_time, _ = (
        model.encoder.cache_aware_stream_step(
            processed_signal=processed,
            processed_signal_length=processed_len,
            cache_last_channel=cache_last_channel,
            cache_last_time=cache_last_time,
            keep_all_outputs=False,
        )
    )

For a complete streaming implementation with microphone input, silence detection, word callbacks, and a FastAPI WebSocket server, see the companion script in the repository files.

Qualitative Examples

The following are exact reference vs. predicted outputs from the validation set — the model transcribed these word-for-word correctly, including full diacritisation (tashkeel):