LateOn

About

What LateOn is

[](https://lighton.ai) [](https://www.linkedin.com/company/lighton/) [](https://x.com/LightOnIO) 📚 Collection | 📝 Blog About the LateOn / DenseOn Family State-of-the-art retrieval is increasingly dominated by closed models, either hidden behind APIs or trained on undisclosed data. This blocks reproducibility, prevents study of possible data leakage, and gatekeeps progress to a handful of private labs. We thus decided to gather and curate a large amount of data and explore various mixtures. We release all the data used in our explorations: - Gathered pre-training data, 1.4B query-documents pairs alongside annotations used for non-destructive filtering (structural filtering, deduplication, cross-encoder pair relevancy) - Best pre-training mixture found with already applied filters - Fine-tuning datasets with query, positive and 2048 mined documents alongside their scores for 1.88M samples. Based on our findings, we trained LateOn (multi-vector/ColBERT) and DenseOn (single vector/dense) models on a proprietary Apache 2.0-compatible training dataset and release those models as well. Both are built on the ModernBERT backbone at 149M parameters, a size we believe sits at the sweet spot: large enough to handle real-world queries and documents, small enough to serve at high throughput in latency-sensitive production systems. For more information, please read our blogpost. LateOn LateOn is a ColBERT (multi-vector) retrieval model built on ModernBERT (149M parameters), trained by LightOn using PyLate. This is the supervised version, fine-tuned with hard-negative contrastive training. The unsupervised version can be found here Notably it: - Beats every existing ColBERT model, including those 4× its size (Jina ColBERT v2 at 559M, Arctic Embed L v2 at 568M). - Holds up under decontamination: when training-overlap samples are stripped from the BEIR corpora, LateOn climbs to 60.36 nDCG@10 on the 12-dataset decontaminated split — first place overall. - Uses fully open data for both pre-training and fine-tuning, with all signals released as metadata so you can rebuild, extend, or replace any filter. This release was focused on contrastive data exploration. We did not run a knowl

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys lateon for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (lateon below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"lateon","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models