Real-time_RE-USE

About

What Real-time_RE-USE is

<span style="color:#76b900;" 🚀+🤫 Real-time RE-USE: Real-time Multilingual Universal Speech Enhancement</span Model Overview Description Real-time RE-USE is a unified, real-time universal speech enhancement framework that explicitly controls both algorithmic and computational latency within a single model. The proposed framework supports 30 distinct latency configurations while maintaining performance close to specialized models, making it easy to adapt to different latency budgets. For Non Real-time applications, check this model = RE-USE In universal speech enhancement, the goal is to restore the quality of diverse degraded speech while preserving fidelity, ensuring that all other factors remain unchanged, e.g., linguistic content, speaker identity, emotion, accent, and other paralinguistic attributes. Inspired by the distortion–perception trade-off theory, our proposed single model achieves a good balance between these two objectives and has the following desirable properties: - Robustness to diverse degradations, including additive noise, reverberation, clipping, bandwidth limitation, codec artifacts, packet loss and low-quality mics . - Support for multiple input sampling rates, including 8, 16, 22.05, 24, 32, 44.1, and 48 kHz. - Strong language-agnostic capability, enabling effective performance across different languages. This model is for research and development only. Usage Directly try our Gradio Interactive Demo by uploading your noisy audio/video !! !! Note that this demo page uses offline inference, so you can easily check the audio quality not the latency. Please refer to onlineinference.py and onlineinference.sh for streaming inference. Environment Setup 1. (For Mamba setup)Pre-built Docker environments can be downloaded here to simplify Mamba setup. 2. If you need bandwidth extension: 3. Download and navigate to the HuggingFace repository: Inference Follow the simple steps below to generate enhanced speech using our model: 1. Place your noisy speech files in the folder noisyaudio/ 2. Run the following command: 3. The enhanced speech files will be saved in offlineenhancedaudio/. That's all ! Note: a. You can enable bandwidth extension by setting th

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How audio & music work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys real-time-re-use for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (real-time-re-use below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -F model="real-time-re-use" -F file=@audio.mp3

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More audio & music