llama-embed-nemotron

Model Overview

Description:

llama-embed-nemotron-8b is a versatile text embedding model trained by NVIDIA and optimized for retrieval, reranking, semantic similarity, and classification use cases. This model has robust capabilities for multilingual and cross-lingual text retrieval. It is designed to serve as a foundational component in text-based Retrieval-Augmented Generation (RAG) systems.

This model achieves state-of-the-art performance on the multilingual MTEB leaderboard as of October 21, 2025.

Together with the model weights, we're releasing the full recipe behind the llama-embed-nemotron-8b:

• A detailed technical report focusing on our Synthetic Data Generation (SDG) pipeline and core design choices.
• The training dataset, featuring a curated mix of public and synthetic data.
• The full training code via the NeMo AutoModel framework.

This model is for non-commercial/research use only.

License/Terms of Use

Governing Terms for llama-embed-nemotron-8b model: NVIDIA License Additional Information: Llama-3.1 Community License Agreement for meta-llama/Llama-3.1-8B. Acceptable Use Policy. Built with Llama.

Team

• Yauhen Babakhin
• Radek Osmulski
• Ronay Ak
• Gabriel Moreira
• Mengyao Xu
• Benedikt Schifferer
• Bo Liu
• Even Oldridge

Correspondence to Yauhen Babakhin (ybabakhin@nvidia.com) and Bo Liu (boli@nvidia.com).

Citation

@misc{babakhin2025llamaembednemotron8buniversaltextembedding,
      title={Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks},
      author={Yauhen Babakhin and Radek Osmulski and Ronay Ak and Gabriel Moreira and Mengyao Xu and Benedikt Schifferer and Bo Liu and Even Oldridge},
      year={2025},
      eprint={2511.07025},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2511.07025},
}

NVIDIA’s Retrieval Models

Deployment Geography:

Global

Use Case:

The llama-embed-nemotron-8b model is intended for researchers developing applications that need to understand or retrieve information from text. It is well-suited for multilingual RAG systems in which queries and documents are textual and may be in different languages.

Release Date:

Hugging Face on 10/21/2025 via https://huggingface.co/nvidia/llama-embed-nemotron-8b

Model Architecture:

• Architecture Type: Transformer Decoder

• Network Architecture: Llama-3.1-8B with bi-directional attention

• This model was developed based on meta-llama/Llama-3.1-8B model.

• Number of model parameters: 7,504,924,672

This llama-embed-nemotron-8b embedding model is a fine-tuned version of Llama-3.1-8B transformer decoder architecture, with a bidirectional attention mechanism. The model consists of 32 hidden layers and an embedding size of 4096, and trained on public datasets and synthetically generated datasets. Embedding models for text retrieval are typically trained using a bi-encoder architecture. This involves encoding a pair of sentences (for example, query and chunked passages) independently using the embedding model. Contrastive learning is used to maximize the similarity between the query and the passage that contains the answer, while minimizing the similarity between the query and sampled negative passages not useful to answer the question.

Input:

| Property | Query