ML-Embed

Model Highlights

• Base architecture: Qwen3-0.6B
• Embedding dimension: 1024
• Sequence embedding: EOS token representation
• Attention type: causal attention
• Trained in two stages (first-stage checkpiont: codefuse-ai/F2LLM-v2-0.6B-Preview)
• Supports multilingual retrieval and semantic similarity
• Supports efficient deployment with:
  • fewer transformer layers
  • factorized embedding matrices (U.pth, V.pth)

Quick Start

With Sentence Transformers

To encode text with the Sentence Transformers library:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "codefuse-ai/ML-Embed-0.6B",
    device="cuda:0",
    model_kwargs={"torch_dtype": "bfloat16"}
)

# Some sample query and documents
query = "What is ML-Embed used for?"
documents = [
    "ML-Embed is a family of multilingual embedding models for retrieval, semantic search, and other NLP tasks.",
    "ML-Embed is trained to produce text embeddings that work well across many languages.",
    "ML-Embed 是 CodeFuse AI 开源的多语言嵌入模型。",
    "ML-Embed — это многоязычная модель эмбеддингов для поиска и семантического сопоставления."
]

# Encode the query and documents separately. The encode_query method uses the query prompt
query_embedding = model.encode_query(query)
document_embeddings = model.encode_document(documents)

print(query_embedding.shape, document_embeddings.shape)
# (1024,) (4, 1024)

# Compute cosine similarity between the query and documents
similarity = model.similarity(query_embedding, document_embeddings)
print(similarity)

With Transformers

Or directly with the Transformers library:

from transformers import AutoModel, AutoTokenizer
import torch
import torch.nn.functional as F

model_path = "codefuse-ai/ML-Embed-0.6B"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModel.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map={"": 0}
)

query = "What is ML-Embed used for?"
query_prompt = "Instruct: Given a question, retrieve passages that can help answer the question.\nQuery: "

documents = [
    "ML-Embed is a family of multilingual embedding models for retrieval, semantic search, and other NLP tasks.",
    "ML-Embed is trained to produce text embeddings that work well across many languages.",
    "ML-Embed 是 CodeFuse AI 开源的多语言嵌入模型。",
    "ML-Embed — это многоязычная модель эмбеддингов для поиска и семантического сопоставления."
]

def encode(sentences):
    batch_size = len(sentences)
    tokenized_inputs = tokenizer(sentences, padding=True, return_tensors="pt").to(model.device)
    last_hidden_state = model(**tokenized_inputs).last_hidden_state
    eos_positions = tokenized_inputs.attention_mask.sum(dim=1) - 1
    embeddings = last_hidden_state[torch.arange(batch_size, device=model.device), eos_positions]
    embeddings = F.normalize(embeddings, p=2, dim=1)
    return embeddings

# Encode the query and documents
query_embedding = encode([query_prompt + query])
document_embeddings = encode(documents)

print(query_embedding.shape, document_embeddings.shape)
# torch.Size([1, 1024]) torch.Size([4, 1024])

# Compute cosine similarity between the query and documents
similarity = query_embedding @ document_embeddings.T
print(similarity)

Prompts

The model supports custom instructions in the following format:

Instruct: your_instruction
Query:

In general, for retrieval and reranking tasks:

• use the prompt for queries
• do not prepend the prompt to documents/passages

For symmetric tasks such as STS, clustering, and bitext mining, you can encode the documents either with or without prompts. The model is trained to support both scenarios.

Efficient Deployment

ML-Embed-0.6B was trained with 3D Matryoshka Learning, including:

• MLL: Matryoshka Layer Learning
• MEL: Matryoshka Embedding Learning
• MRL: Matryoshka Representation Learning

This enables multiple deployment modes.

1. Compatibility Mode

The default released checkpoint is a standard Qwen3-compatible model. You can load it normally with AutoModel or SentenceTransformer without any code changes (refer to the examples above).

This is the recommended option if you want the simplest integration.

2. Fewer-Layer Deployment with MLL

This model was trained so that shallower versions remain useful. If you want to save memory or compute, you can deploy a smaller model by editing:

• num_hidden_layers
• max_window_layers

in config.json to a value smaller than the current one.

For example, changing both values from 28 to 16 will make transformers load only the first 16 layers and ignore the remaining weights.

This works automatically with the Hugging Face transformers library.

Note: make sure num_hidden_layers and max_window_layers stay consistent. If you are using transformers v5, you will also need to trucate layer_types in the config file according to the new