nemo-nano-codec-22khz-1.89kbps-21.5fps

About

What nemo-nano-codec-22khz-1.89kbps-21.5fps is

NVIDIA NeMo NanoCodec img{ display: inline-table; vertical-align: small; margin: 0; padding: 0; } [](#model-architecture) The NeMo NanoCodec is a neural audio codec that leverages finite scalar quantization and adversarial training with large speech language models to achieve state-of-the-art audio compression across different bitrate and frame rate ranges. Model variant details: This model is ready for commercial/non-commercial use. NeMo NanoCodec variants Model | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Embed Dim | FSQ Levels | :-----------:|:-----------:|:----------:|:----------:|:-----------:|:-------------:|:-----------:|:------------:| 1.78kbps-12.5fps| 22050 | 12.5 | 1.78kpbs | 13 | 2016 | 52 | [8, 7, 6, 6] | 0.6kbps-12.5fps | 22050 | 12.5 | 0.6kpbs | 4 | 4032 | 16 | [9, 8, 8, 7] | 1.89kbps-21.5fps | 22050 | 21.5 | 1.89kpbs | 8 | 2016 | 32 | [8, 7, 6, 6] | ⚠️ Note on 0.6kbps-12.5fps This variant is designed for fine-tuning with a limited set of speakers, as shown in our S2S Duplex paper. It is not recommended for general-purpose audio encoding or decoding. ℹ️ Recommended Variants Both 1.78kbps-12.5fps and 1.89kbps-21.5fps achieve similar audio reconstruction quality. However, our Magpie TTS model performs best with 1.89kbps-21.5fps. License/Terms of Use NVIDIA Open Model License Agreement Deployment Geography: Use Case: Release Date: Model Architecture NeMo NanoCodec is composed of a fully convolutional generator neural network and three discriminators. The generator comprises an encoder, followed by vector quantization, and a HiFi-GAN-based decoder. The non-causal encoder consists of five residual blocks, each block containing three residual layers similar to the multi-receptive field fusion (MRF) module. The causal decoder, based on the HiFi-GAN vocoder, uses upsampling rates that are the reverse of the encoder's One-Dimensional (1D) convolutional strides. For the vector quantization, we have used Finite Scalar Quantization (FSQ) with thirteen codebooks and four dimensions per code and 2016 codes per codebook. For the discriminators, we utilize three neural networks, all employing a squared-GAN and feature-matching loss. We ad

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys nemo-nano-codec-22khz-1-89kbps-21-5fps for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (nemo-nano-codec-22khz-1-89kbps-21-5fps below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"nemo-nano-codec-22khz-1-89kbps-21-5fps","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More embedding models