Model reference · open weights
msmarco-distilbert-dot is an open-weight embedding model from sentence-transformers, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
msmarco-distilbert-dot-v5 This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and was designed for semantic search. It has been trained on 500K (query, answer) pairs from the MS MARCO dataset. For an introduction to semantic search, have a look at: SBERT.net - Semantic Search Usage (Sentence-Transformers) Using this model becomes easy when you have sentence-transformers installed: Then you can use the model like this: Usage (HuggingFace Transformers) Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the correct pooling-operation on-top of the contextualized word embeddings. Technical Details In the following some technical details how this model must be used: Training See trainscript.py in this repository for the used training script. The model was trained with the parameters: DataLoader: torch.utils.data.dataloader.DataLoader of length 7858 with parameters: Loss: sentencetransformers.losses.MarginMSELoss.MarginMSELoss Parameters of the fit()-Method: Full Model Architecture Citing & Authors This model was trained by sentence-transformers. If you find this model helpful, feel free to cite our publication Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks: License This model is released under the Apache 2 license. However, note that this model was trained on the MS MARCO dataset which has it's own license restrictions: MS MARCO - Terms and Conditions.
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | sentence-transformers |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 66M |
| Context | 512 tokens |
| Variants | 1 |
| Runs with | sentence-transformers |
| Released | 2022-03-02 |
| Popularity | 26k downloads / month |
| Likes | 15 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| msmarco-distilbert-dot-v5 | 66M | BF16 | ~0.2 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys msmarco-distilbert-dot for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (msmarco-distilbert-dot below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"msmarco-distilbert-dot","input":"text to embed"}'
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗
Explore