mLateOn-unsupervised

About

What mLateOn-unsupervised is

[](https://lighton.ai) [](https://www.linkedin.com/company/lighton/) [](https://x.com/LightOnIO) 📚 Collection | 📝 Multilingual Blog | 📝 English Blog | 📝 Paper 🎯 TL;DR: The intermediate multilingual ColBERT checkpoint produced by Stage 1 only (unsupervised contrastive pre-training) of the mLateOn pipeline, trained on a multilingual dataset with 2.8B query–document pairs across nine languages (including 25% cross-lingual pairs). Released as a strong starting point for your own supervised fine-tuning, knowledge distillation, or downstream adaptation. About the mDenseOn / mLateOn Family With DenseOn and LateOn, we demonstrated that an open, carefully curated data recipe can match closed-data retrieval models on English. mDenseOn and mLateOn extend this recipe to multilingual, long-context, and code retrieval. Rather than independently collecting multilingual corpora from scratch (which would be expensive, uneven across languages, and hard to curate at the same quality), we applied the translate-train approach: machine-translating our validated English data into eight target languages (French, German, Italian, Spanish, Portuguese, Swedish, Norwegian, and Arabic) and adding cross-lingual pairs for cross-lingual alignment. For more information, please read our multilingual models blog post, our English models blog post and our paper. mLateOn-unsupervised mLateOn-unsupervised is the output of the first stage of the mLateOn training pipeline. It has been pre-trained on a large, filtered multilingual corpus of approximately 2.8B query–document pairs across nine languages (including 25% cross-lingual pairs) using in-batch contrastive learning, but has not yet been fine-tuned with mined hard negatives. For most production use cases, you should use the fully-trained mLateOn instead. This unsupervised checkpoint is intended for: - Researchers studying what each pipeline stage contributes in a multilingual setting - Practitioners who want to fine-tune on their own domain-specific or language-specific data - Distillation experiments where you want to start from a strong but un-aligned multilingual base - Anyone running their own ablations on hard-negative mining strategies or t

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys mlateon-unsupervised for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mlateon-unsupervised below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"mlateon-unsupervised","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models