rubaistt_v2_medium

Whitepaper

For more details on the methodology and research behind this model, visit: https://uz-speech.web.app/rubaistt02m

Training and filtering code: https://github.com/Islomov49/rubaistt_v2-open-sourced

Support my works and open-source movement: https://tirikchilik.uz/islomovs

Model Details

• Base Model: Whisper Medium
• Parameters: 769M
• Performance:
  • WER: ~17%
  • CER: ~5.5%

Training Data

This model was fine-tuned on approximately 475 hours of diverse Uzbek audio data including:

• Common Voice 17 dataset (filtered)
• USC (filtered)
• Google fleurs (filtered)
• Podcasts Tashkent Dialect Youtube Uzbek Speech Dataset: Link HF
• News Youtube Uzbek Speech Dataset: Link HF
• IT Youtube Uzbek Speech Dataset: Link HF

The dataset consisted of 50% human-transcribed and 50% pseudo-transcribed material (using Gemini 2.5 Pro). Special attention was given to Tashkent dialect audio materials to ensure strong performance on this dialect.

A technique was used to filter out datasets based on Word Error Rate (WER) and similarity checks. The script for this process will also be open-sourced.

Usage Example

import torch
import torchaudio
from transformers import WhisperProcessor, WhisperForConditionalGeneration

# Load model and processor
processor = WhisperProcessor.from_pretrained("islomov/rubaistt_v2_medium")
model = WhisperForConditionalGeneration.from_pretrained("islomov/rubaistt_v2_medium")

def transcribe_audio(audio_path):

    global model, processor

    # Move to GPU if available
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model = model.to(device)

    # Load and preprocess audio
    waveform, sample_rate = torchaudio.load(audio_path)
    if sample_rate != 16000:
        waveform = torchaudio.functional.resample(waveform, sample_rate, 16000)

    # Convert to mono if needed
    if waveform.shape[0] > 1:
        waveform = waveform.mean(dim=0, keepdim=True)

    # Process audio
    input_features = processor(
        waveform.squeeze().numpy(),
        sampling_rate=16000,
        return_tensors="pt",
        language="uz"
    ).input_features.to(device)

    # Generate transcription
    with torch.no_grad():
        predicted_ids = model.generate(input_features)

    # Decode
    transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
    return transcription

# Example usage
if __name__ == "__main__":
    audio_file = "some_audio_max_30_sec.wav"

    text = transcribe_audio(audio_file)
    print(f"Transcription: {text}")

Future Improvements

Future versions will include more regional Uzbek dialects to improve overall coverage.