multilingual-e5-large

About

What multilingual-e5-large is

Multilingual-E5-large Multilingual E5 Text Embeddings: A Technical Report. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 24 layers and the embedding size is 1024. Usage Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. Supported Languages This model is initialized from xlm-roberta-large and continually trained on a mixture of multilingual datasets. It supports 100 languages from xlm-roberta, but low-resource languages may see performance degradation. Training Details Initialization: xlm-roberta-large First stage: contrastive pre-training with weak supervision Second stage: supervised fine-tuning For all labeled datasets, we only use its training set for fine-tuning. For other training details, please refer to our paper at https://arxiv.org/pdf/2402.05672. Benchmark Results on Mr. TyDi MTEB Benchmark Evaluation Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark. Support for Sentence Transformers Below is an example for usage with sentencetransformers. Package requirements pip install sentencetransformers~=2.2.2 Contributors: michaelfeil FAQ 1. Do I need to add the prefix "query: " and "passage: " to input texts? Yes, this is how the model is trained, otherwise you will see a performance degradation. Here are some rules of thumb: - Use "query: " and "passage: " correspondingly for asymmetric tasks such as passage retrieval in open QA, ad-hoc information retrieval. - Use "query: " prefix for symmetric tasks such as semantic similarity, bitext mining, paraphrase retrieval. - Use "query: " prefix if you want to use embeddings as features, such as linear probing classification, clustering. 2. Why are my reproduced results slightly different from reported in the model card? Different versions of transformers and pytorch could cause negligible but non-zero performance differences. 3. Why does the cosine similarity scores distribute around 0.7 to 1.0? This is a known and expected behavior as we use a low temperature 0.01 for InfoNCE contrastive loss. For text embedding tasks like text retrieval or semantic similarity, what matters is the re

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Benchmarks

Reported results

As published on the model card — the maker's own numbers, not measured by AxForge.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys multilingual-e5-large for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (multilingual-e5-large below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"multilingual-e5-large","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models