encodec_24khz

About

What encodec_24khz is

Model Card for EnCodec This model card provides details and information about EnCodec, a state-of-the-art real-time audio codec developed by Meta AI. Model Details Model Description EnCodec is a high-fidelity audio codec leveraging neural networks. It introduces a streaming encoder-decoder architecture with quantized latent space, trained in an end-to-end fashion. The model simplifies and speeds up training using a single multiscale spectrogram adversary that efficiently reduces artifacts and produces high-quality samples. It also includes a novel loss balancer mechanism that stabilizes training by decoupling the choice of hyperparameters from the typical scale of the loss. Additionally, lightweight Transformer models are used to further compress the obtained representation while maintaining real-time performance. - Developed by: Meta AI - Model type: Audio Codec Model Sources - Repository: GitHub Repository - Paper: EnCodec: End-to-End Neural Audio Codec Uses Direct Use EnCodec can be used directly as an audio codec for real-time compression and decompression of audio signals. It provides high-quality audio compression and efficient decoding. The model was trained on various bandwiths, which can be specified when encoding (compressing) and decoding (decompressing). Two different setup exist for EnCodec: - Non-streamable: the input audio is split into chunks of 1 seconds, with an overlap of 10 ms, which are then encoded. - Streamable: weight normalizationis used on the convolution layers, and the input is not split into chunks but rather padded on the left. Downstream Use EnCodec can be fine-tuned for specific audio tasks or integrated into larger audio processing pipelines for applications such as speech generation, music generation, or text to speech tasks. [More Information Needed] How to Get Started with the Model Use the following code to get started with the EnCodec model using a dummy example from the LibriSpeech dataset (~9MB). First, install the required Python packages: Then load an audio sample, and run a forward pass of the model: Training Details The model was trained for 300 epochs, with one epoch being 2,000 updates with the Adam optimizer with a

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys encodec-24khz for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (encodec-24khz below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"encodec-24khz","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More embedding models