e5-mistral-bnb

About

What e5-mistral-bnb is

intfloat/e5-mistral-7b-instruct (Quantized) Description This model is a quantized version of the original model intfloat/e5-mistral-7b-instruct. It's quantized using the BitsAndBytes library to 4-bit using the bnb-my-repo space. Quantization Details - Quantization Type: int4 - bnb4bitquanttype: nf4 - bnb4bitusedoublequant: True - bnb4bitcomputedtype: bfloat16 - bnb4bitquantstorage: uint8 📄 Original Model Information E5-mistral-7b-instruct Improving Text Embeddings with Large Language Models. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 32 layers and the embedding size is 4096. Usage Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. Sentence Transformers Have a look at configsentencetransformers.json for the prompts that are pre-configured, such as websearchquery, stsquery, and summarizationquery. Additionally, check out unilm/e5/utils.py for prompts we used for evaluation. You can use these via e.g. model.encode(queries, prompt="Instruct: Given a claim, find documents that refute the claim\nQuery: "). Transformers Supported Languages This model is initialized from Mistral-7B-v0.1 and fine-tuned on a mixture of multilingual datasets. As a result, it has some multilingual capability. However, since Mistral-7B-v0.1 is mainly trained on English data, we recommend using this model for English only. For multilingual use cases, please refer to multilingual-e5-large. MTEB Benchmark Evaluation Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark. FAQ 1. Do I need to add instructions to the query? Yes, this is how the model is trained, otherwise you will see a performance degradation. The task definition should be a one-sentence instruction that describes the task. This is a way to customize text embeddings for different scenarios through natural language instructions. Please check out unilm/e5/utils.py for instructions we used for evaluation. On the other hand, there is no need to add instructions to the document side. 2. Why are my reproduced results slightly different from reported in the model card? Different versions of transformers and

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Benchmarks

Reported results

As published on the model card — the maker's own numbers, not measured by AxForge.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys e5-mistral-bnb for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (e5-mistral-bnb below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"e5-mistral-bnb","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models