LFM2-ColBERT

src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" />

LFM2-ColBERT-350M

LFM2-ColBERT-350M is a late interaction retriever with excellent multilingual performance. It allows you to store documents in one language (for example, a product description in English) and retrieve them in many languages with high accuracy.

• LFM2-ColBERT-350M offers best-in-class accuracy across different languages.
• Inference speed is on par with models 2.3 times smaller, thanks to the efficient LFM2 backbone.
• You can use it as a drop-in replacement in your current RAG pipelines to improve performance.

Find more information about LFM2-ColBERT-350M in our blog post.

[!NOTE] 🚀 Try our demo: https://huggingface.co/spaces/LiquidAI/LFM2-ColBERT

📄 Model details

Late interaction retrievers like LFM2-ColBERT-350M are particularly interesting because they preserve much of the expressivity of re-rankers while retaining the efficiency of bi-encoders. In practice, they're used to both retrieve documents at scale (like bi-encoders) and rank them at the same time (like rerankers).

We recommend using this model for various RAG use cases, such as:

• E-commerce: Find products across many languages with semantic search at scale.
• On-device semantic search: Ask questions to your phone in natural language to retrieve files, emails, and notes.
• Enterprise knowledge assistants: Retrieve internal legal, financial, and technical documents in different languages.

Document length: 512 tokens

Query length: 32 tokens

Output dimensionality: 128 tokens

Similarity function: MaxSim

Supported languages: English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish.

ColBERT(
  (0): Transformer({'max_seq_length': 511, 'do_lower_case': False}) with Transformer model: Lfm2Model
  (1): Dense({'in_features': 1024, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

🏃 How to run

Sentence Transformers

This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:

pip install "sentence-transformers>=6.0.0"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("LiquidAI/LFM2-ColBERT-350M")

query = "Which planet is known as the Red Planet?"
documents = [
    "Venus wird oft als Zwilling der Erde bezeichnet, wegen ihrer ähnlichen Größe.",
    "Mars, connue pour son apparence rougeâtre, est souvent appelée la planète rouge.",
    "Júpiter es el planeta más grande del sistema solar.",
    "Saturno è famoso per i suoi bellissimi anelli.",
]

query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# (32, 128) (17, 128)

# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[30.3632, 30.4892, 30.4024, 30.3004]])

PyLate

First, install the PyLate and transformers library:

pip install -U pylate

Retrieval

Use this model with PyLate to index and retrieve documents. The index uses FastPLAID for efficient similarity search.

Indexing documents

Load LFM2-ColBERT-350M and initialize the PLAID index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2-ColBERT-350M",
)
model.tokenizer.pad_token = model.tokenizer.eos_token

# Step 2: Initialize the PLAID index
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)