tadabur-embedding

Use cases

• Retrieval / semantic search — find the most similar recitations to a query clip.
• Deduplication — detect near-duplicate audio across large collections.
• Clustering — group recordings by reciter, style, or acoustic similarity.
• Classification — use embeddings as features for downstream classifiers.
• Frame analysis — use the per-frame embedding sequence (before pooling) for tasks that need temporal detail: localizing acoustic events within a clip, aligning or segmenting recitations, and detecting variations over time.

Requirements

transformers == 4.40
torch
torchaudio

If you hit AttributeError: '...Model' object has no attribute 'all_tied_weights_keys' (or similar) on from_pretrained, you're on an older cached copy of this repo's code — transformers versions 5.x call self.post_init()-dependent bookkeeping during loading that earlier revisions of this wrapper didn't set up. This has been fixed; clearing your local transformers_modules cache for this repo and re-downloading it again.

Quick start

import torch
import torchaudio
from transformers import AutoModel

model = AutoModel.from_pretrained("FaisaI/tadabur-embedding", trust_remote_code=True).eval()

# --- Load audio (16 kHz mono) ---
waveform, sr = torchaudio.load("recitation.wav")
waveform = waveform.mean(dim=0, keepdim=True)  # mono
if sr != 16000:
    waveform = torchaudio.functional.resample(waveform, sr, 16000)

# --- Log-mel spectrogram (EAT preprocessing) ---
waveform = waveform - waveform.mean()
mel = torchaudio.compliance.kaldi.fbank(
    waveform,
    htk_compat=True,
    sample_frequency=16000,
    use_energy=False,
    window_type="hanning",
    num_mel_bins=128,
    dither=0.0,
    frame_shift=10,
)  # (n_frames, 128)

# Pad or truncate to 1024 frames (= 10.24 s)
target_length = 1024
n_frames = mel.shape[0]
if n_frames < target_length:
    mel = torch.nn.functional.pad(mel, (0, 0, 0, target_length - n_frames))
else:
    mel = mel[:target_length]

# Normalize with tadabur dataset statistics
norm_mean, norm_std = -4.381, 3.628
mel = (mel - norm_mean) / (norm_std * 2)
mel = mel[None, None]  # (1, 1, 1024, 128)

# --- Extract embeddings ---
with torch.no_grad():
    semantic = model.semantic_embedding(mel)  # (1, 384) L2-normalized, ayah content
    speaker = model.speaker_embedding(mel)    # (1, 128) L2-normalized, reciter identity

    features = model.extract_features(mel)    # (1, 513, 768) = CLS + 512 frame patches
    frame_embeddings = features[:, 1:]        # (1, 512, 768) frame-level (~50 Hz)

For retrieval / search / deduplication, compare semantic_embedding vectors with a dot product (they are L2-normalized, so this is cosine similarity). For reciter similarity, use speaker_embedding the same way. The raw encoder features from extract_features are best for frame-level temporal tasks and as input to downstream models.

For longer audio, split into 10.24 s segments and embed each one. Frame-level embeddings (features[:, 1:]) preserve temporal order and can be used directly for alignment and localization tasks.

Training data

Trained on FaisaI/tadabur, a dataset of Quranic recitation audio, in two stages: EAT self-supervised pretraining of the encoder, followed by multi-axis contrastive training (semantic axis aligned to ayah text embeddings, speaker axis trained on reciter identity with guaranteed same-reciter pairs). Spectrogram normalization statistics (norm_mean = -4.381, norm_std = 3.628) were computed on this dataset — use them (not the AudioSet defaults) when preprocessing.

Intended use & limitations

• Best suited for recitation-style Arabic speech audio; performance on unrelated audio domains is not guaranteed.
• Input is expected as 16 kHz mono audio converted to 128-bin log-mel spectrograms, in windows of up to 10.24 s.

License

Released under CC BY-NC 4.0 — free for non-commercial use with attribution.

Acknowledgements

The model architecture and pretraining recipe follow EAT: Self-Supervised Pre-Training with Efficient Audio Transformer (Chen et al., 2024). The Hugging Face wrapper code is adapted from worstchan/EAT-base_epoch30_pretrain.

Citation

@misc{tadabur-embedding,
  title  = {tadabur-embedding: audio embeddings for Quranic recitation},
  author = {Faisal},
  year   = {2026},
  url    = {https://huggingface.co/FaisaI/tadabur-embedding}
}