mDenseOn

About

What mDenseOn is

[](https://lighton.ai) [](https://www.linkedin.com/company/lighton/) [](https://x.com/LightOnIO) 📚 Collection | 📝 Multilingual Blog | 📝 English Blog | 📝 Paper 🎯 TL;DR: A 307M-parameter multilingual dense (single-vector) retrieval model achieving state-of-the-art results across multilingual retrieval (MIRACL), long-document retrieval (MLDR), English general-domain retrieval (BEIR), and code retrieval (MTEB Code). Built by extending our validated open English data recipe to eight additional languages via translate-train, producing one of the largest open multilingual retrieval training sets to date (2.8B pairs). About the mDenseOn / mLateOn Family With DenseOn and LateOn, we demonstrated that an open, carefully curated data recipe can match closed-data retrieval models on English. mDenseOn and mLateOn extend this recipe to multilingual, long-context, and code retrieval. Rather than independently collecting multilingual corpora from scratch (which would be expensive, uneven across languages, and hard to curate at the same quality), we applied the translate-train approach: machine-translating our validated English data into eight target languages (French, German, Italian, Spanish, Portuguese, Swedish, Norwegian, and Arabic) and adding cross-lingual pairs for cross-lingual alignment. For more information, please read our multilingual models blog post, our English models blog post and our paper. mDenseOn mDenseOn is a multilingual dense (single-vector) retrieval model built on mmBERT-base (307M parameters), trained by LightOn. It encodes queries and documents independently using cosine similarity with query:/document: prefixes and [CLS] pooling, supporting context lengths of up to 8,192 tokens. mDenseOn notably: - Achieves 56.70 NDCG@10 on BEIR, slightly surpassing the English-only DenseOn (56.20), confirming that multilingual capabilities do not come at the cost of English quality. - Reaches 59.61 on MIRACL target languages, competitive with similarly-sized models like EmbeddingGemma. - Scores 64.98 on MLDR target languages, competitive with most dense baselines - Achieves 71.53 on MTEB Code, the second-best sub-350M dense model, despite using only fine-tuning-stage

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys mdenseon for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mdenseon below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"mdenseon","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models