static-similarity-mrl-multilingual

This is a sentence-transformers model trained on the wikititles, tatoeba, talks, europarl, global_voices, muse, wikimatrix, opensubtitles, stackexchange, quora, wikianswers_duplicates, all_nli, simple_wiki, altlex, flickr30k_captions, coco_captions, nli_for_simcse and negation datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, paraphrase mining, text classification, clustering, and more.

Read our Static Embeddings blogpost to learn more about this model and how it was trained.

• 0 Active Parameters: This model does not use any active parameters, instead consisting exclusively of averaging pre-computed token embeddings.
• 100x to 400x faster: On CPU, this model is 100x to 400x faster than common options like multilingual-e5-small. On GPU, it's 10x to 25x faster.
• Matryoshka: This model was trained with a Matryoshka loss, allowing you to truncate the embeddings for faster retrieval at minimal performance costs.
• Evaluations: See Evaluations for details on performance on NanoBEIR, embedding speed, and Matryoshka dimensionality truncation.
• Training Script: See train.py for the training script used to train this model from scratch.

See static-retrieval-mrl-en-v1 for an English static embedding model that has been finetuned specifically for retrieval tasks.

Model Details

Model Description

• Model Type: Sentence Transformer

• Maximum Sequence Length: inf tokens

• Output Dimensionality: 1024 dimensions

• Similarity Function: Cosine Similarity

• Training Datasets:

  • wikititles
  • tatoeba
  • talks
  • europarl
  • global_voices
  • muse
  • wikimatrix
  • opensubtitles
  • stackexchange
  • quora
  • wikianswers_duplicates
  • all_nli
  • simple_wiki
  • altlex
  • flickr30k_captions
  • coco_captions
  • nli_for_simcse
  • negation

• Languages: en, multilingual, ar, bg, ca, cs, da, de, el, es, et, fa, fi, fr, gl, gu, he, hi, hu, hy, id, it, ja, ka, ko, ku, lt, lv, mk, mn, mr, ms, my, nb, nl, pl, pt, ro, ru, sk, sl, sq, sr, sv, th, tr, uk, ur, vi, zh, hr

• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): StaticEmbedding(
    (embedding): EmbeddingBag(105879, 1024, mode='mean')
  )
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Downloa