mDenseOn

📚 Collection | 📝 Multilingual Blog | 📝 English Blog | 📝 Paper

🎯 TL;DR: A 307M-parameter multilingual dense (single-vector) retrieval model achieving state-of-the-art results across multilingual retrieval (MIRACL), long-document retrieval (MLDR), English general-domain retrieval (BEIR), and code retrieval (MTEB Code). Built by extending our validated open English data recipe to eight additional languages via translate-train, producing one of the largest open multilingual retrieval training sets to date (2.8B pairs).

About the mDenseOn / mLateOn Family

With DenseOn and LateOn, we demonstrated that an open, carefully curated data recipe can match closed-data retrieval models on English. mDenseOn and mLateOn extend this recipe to multilingual, long-context, and code retrieval.

Rather than independently collecting multilingual corpora from scratch (which would be expensive, uneven across languages, and hard to curate at the same quality), we applied the translate-train approach: machine-translating our validated English data into eight target languages (French, German, Italian, Spanish, Portuguese, Swedish, Norwegian, and Arabic) and adding cross-lingual pairs for cross-lingual alignment.

For more information, please read our multilingual models blog post, our English models blog post and our paper.

mDenseOn

mDenseOn is a multilingual dense (single-vector) retrieval model built on mmBERT-base (307M parameters), trained by LightOn. It encodes queries and documents independently using cosine similarity with query:/document: prefixes and [CLS] pooling, supporting context lengths of up to 8,192 tokens.

mDenseOn notably:

• Achieves 56.70 NDCG@10 on BEIR, slightly surpassing the English-only DenseOn (56.20), confirming that multilingual capabilities do not come at the cost of English quality.
• Reaches 59.61 on MIRACL target languages, competitive with similarly-sized models like EmbeddingGemma.
• Scores 64.98 on MLDR target languages, competitive with most dense baselines
• Achieves 71.53 on MTEB Code, the second-best sub-350M dense model, despite using only fine-tuning-stage code data and no code-specific pre-training.

Alongside mDenseOn, we also trained mLateOn, a late-interaction variant using the same setup. mLateOn achieves substantially stronger results, especially on multilingual and long-context tasks, and generalizes to languages outside of the training set, effectively circumventing the main limitation of translate-train for dense models. If your use case permits multi-vector retrieval, we recommend mLateOn.

See our multilingual blog post for full results and analysis.

Results

Headline Results (NDCG@10)

MIRACLtgt and MLDRtgt include only the languages overlapping with our target languages; MIRACL and MLDR include all benchmark languages.

mDenseOn at 307M parameters is competitive with or outperforms models up to twice its size on most target-language axes. Among dense models of comparable size, it outperforms EmbeddingGemma and granite-311m-r2 on every target-language axis, and gte-multilingual-base on all but MLDR.

Key Findings

English quality improved. mDenseOn reaches 56.70 on BEIR, slightly surpassing the English-only DenseOn (56.20), confirming that multilingual training need not degrade English performance.

Strong target-language performance. On the nine languages we translated into, mDenseOn is competitive with models trained on many more languages.

Dense models generalize less to unseen languages. On the full MIRACL and MLDR benchmarks (which include l