Covo-Audio

About

What Covo-Audio is

Covo-Audio Covo-Audio Technical Report [](https://arxiv.org/abs/2602.09823) [](https://github.com/Tencent/Covo-Audio) [](https://huggingface.co/tencent/Covo-Audio-Chat) 📖 Overview Covo-Audio is a 7B-parameter end-to-end large audio language model that directly processes continuous audio inputs and generates audio outputs within a single unified architecture, which is presented in the paper Covo-Audio Technical Report. We release Covo-Audio-Chat in this repository. Key Features - Hierarchical Tri-modal Speech-Text Interleaving: We propose a framework designed to achieve deep alignment and fusion across modalities and scales. The Tri-modal aspect integrates continuous acoustic features, discrete speech tokens, and natural language text within a unified sequence, effectively bridging the gap between high-fidelity prosodic nuances and robust semantic structures. - Mitigating Intelligence-Speaker Coupling: We propose a intelligence-speaker decoupling technique that decouples speaker from dialogue intelligence via multi-speaker training, then develop a contextual adaptation method to transfer and share high-quality TTS voice. - Native Full-Duplex Voice Interaction: We evolve Covo-Audio into Covo-Audio-Chat-FD, a variant with native, low-latency full-duplex capability. - Comprehensive State-of-the-Art Performance: Achieving state-of-the-art or competitive performance among models of comparable scale across a broad spectrum of tasks, including spoken dialogue, speech understanding, audio understanding, and full-duplex voice interaction. 🔧 Installation 1. Requirements Recommends Python = 3.11 2. Clone Repository 3. Download Pretrained Models Using HuggingFace: By running the above script, you can use the model downloaded from huggingface to override the directory of the same name in this repository. Or you can specify your own directory to store the model by modifying the local-dir argument (In this case, you need to edit the arguments modeldir and decodeloadpath in example.sh accordingly before running the inference script). 🚀 Usage Run Inference Scripts After completeing the configuration and model downloading, you can perform one-click inference by running the script:

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How audio & music work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys covo-audio for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (covo-audio below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -F model="covo-audio" -F file=@audio.mp3

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More audio & music