Model reference · open weights
text2vec-multilingual is an open-weight embedding model from shibing624. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | shibing624 |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Parameters (lead) | 118M |
| Context | 512 tokens |
| Runs with | sentence-transformers |
| Released | 2023-06-22 |
| Popularity | 127k downloads / month |
| Licence | Open weights |
About
This is a CoSENT(Cosine Sentence) model: shibing624/text2vec-base-multilingual.
It maps sentences to a 384 dimensional dense vector space and can be used for tasks like sentence embeddings, text matching or semantic search.
For an automated evaluation of this model, see the Evaluation Benchmark: text2vec
Available languages are: de, en, es, fr, it, nl, pl, pt, ru, zh
| Arch | BaseModel | Model | ATEC | BQ | LCQMC | PAWSX | STS-B | SOHU-dd | SOHU-dc | Avg | QPS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Word2Vec | word2vec | w2v-light-tencent-chinese | 20.00 | 31.49 | 59.46 | 2.57 | 55.78 | 55.04 | 20.70 | 35.03 | 23769 |
| SBERT | xlm-roberta-base | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 | 18.42 | 38.52 | 63.96 | 10.14 | 78.90 | 63.01 | 52.28 | 46.46 | 3138 |
| Instructor | hfl/chinese-roberta-wwm-ext | moka-ai/m3e-base | 41.27 | 63.81 | 74.87 | 12.20 | 76.96 | 75.83 | 60.55 | 57.93 | 2980 |
| CoSENT | hfl/chinese-macbert-base | shibing624/text2vec-base-chinese | 31.93 | 42.67 | 70.16 | 17.21 | 79.30 | 70.27 | 50.42 | 51.61 | 3008 |
| CoSENT | hfl/chinese-lert-large | GanymedeNil/text2vec-large-chinese | 32.61 | 44.59 | 69.30 | 14.51 | 79.44 | 73.01 | 59.04 | 53.12 | 2092 |
| CoSENT | nghuyong/ernie-3.0-base-zh | shibing624/text2vec-base-chinese-sentence | 43.37 | 61.43 | 73.48 | 38.90 | 78.25 | 70.60 | 53.08 | 59.87 | 3089 |
| CoSENT | nghuyong/ernie-3.0-base-zh | shibing624/text2vec-base-chinese-paraphrase | 44.89 | 63.58 | 74.24 | 40.90 | 78.93 | 76.70 | 63.30 | 63.08 | 3066 |
| CoSENT | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 | shibing624/text2vec-base-multilingual | 32.39 | 50.33 | 65.64 | 32.56 | 74.45 | 68.88 | 51.17 | 53.67 | 4004 |
说明：
shibing624/text2vec-base-chinese模型，是用CoSENT方法训练，基于hfl/chinese-macbert-base在中文STS-B数据训练得到，并在中文STS-B测试集评估达到较好效果，运行examples/training_sup_text_matching_model.py代码可训练模型，模型文件已经上传HF model hub，中文通用语义匹配任务推荐使用shibing624/text2vec-base-chinese-sentence模型，是用CoSENT方法训练，基于nghuyong/ernie-3.0-base-zh用人工挑选后的中文STS数据集shibing624/nli-zh-all/text2vec-base-chinese-sentence-dataset训练得到，并在中文各NLI测试集评估达到较好效果，运行examples/training_sup_text_matching_model_jsonl_data.py代码可训练模型，模型文件已经上传HF model hub，中文s2s(句子vs句子)语义匹配任务推荐使用shibing624/text2vec-base-chinese-paraphrase模型，是用CoSENT方法训练，基于nghuyong/ernie-3.0-base-zh用人工挑选后的中文STS数据集shibing624/nli-zh-all/text2vec-base-chinese-paraphrase-dataset，数据集相对于shibing624/nli-zh-all/text2vec-base-chinese-sentence-dataset加入了s2p(sentence to paraphrase)数据，强化了其长文本的表征能力，并在中文各NLI测试集评估达到SOTA，运行examples/training_sup_text_matching_model_jsonl_data.py代码可训练模型，模型文件已经上传HF model hub，中文s2p(句子vs段落)语义匹配任务推荐使用shibing624/text2vec-base-multilingual模型，是用CoSENT方法训练，基于sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2用人工挑选后的多语言STS数据集shibing624/nli-zh-all/text2vec-base-multilingual-dataset训练得到，并在中英文测试集评估相对于原模型效果有提升，运行examples/training_sup_text_matching_model_jsonl_data.py代码可训练模型，模型文件已经上传HF model hub，多语言语义匹配任务推荐使用w2v-light-tencent-chinese是腾讯词向量的Word2Vec模型，CPU加载使用，适用于中文字面匹配任务和缺少数据的冷From the published model card. Full card on the HuggingFace links in the sidebar.
Benchmarks
As published on the model card — the maker's own numbers, not measured by AxForge.
| Task | Dataset | Metric | Score |
|---|---|---|---|
| Classification | MTEB AmazonCounterfactualClassification (en) | accuracy | 70.970 |
| Classification | MTEB AmazonCounterfactualClassification (en) | ap | 33.952 |
| Classification | MTEB AmazonCounterfactualClassification (en) | f1 | 65.147 |
| Classification | MTEB AmazonCounterfactualClassification (de) | accuracy | 68.694 |
| Classification | MTEB AmazonCounterfactualClassification (de) | ap | 79.683 |
| Classification | MTEB AmazonCounterfactualClassification (de) | f1 | 66.550 |
| Classification | MTEB AmazonCounterfactualClassification (en-ext) | accuracy | 70.907 |
| Classification | MTEB AmazonCounterfactualClassification (en-ext) | ap | 20.748 |
| Classification | MTEB AmazonCounterfactualClassification (en-ext) | f1 | 58.644 |
| Classification | MTEB AmazonCounterfactualClassification (ja) | accuracy | 61.606 |
| Classification | MTEB AmazonCounterfactualClassification (ja) | ap | 14.136 |
| Classification | MTEB AmazonCounterfactualClassification (ja) | f1 | 49.980 |
| Classification | MTEB AmazonPolarityClassification | accuracy | 66.103 |
| Classification | MTEB AmazonPolarityClassification | ap | 61.101 |
| Classification | MTEB AmazonPolarityClassification | f1 | 65.752 |
| Classification | MTEB AmazonReviewsClassification (en) | accuracy | 33.134 |
| Classification | MTEB AmazonReviewsClassification (en) | f1 | 32.791 |
| Classification | MTEB AmazonReviewsClassification (de) | accuracy | 33.388 |
| Classification | MTEB AmazonReviewsClassification (de) | f1 | 33.191 |
| Classification | MTEB AmazonReviewsClassification (es) | accuracy | 34.824 |
| Classification | MTEB AmazonReviewsClassification (es) | f1 | 34.297 |
| Classification | MTEB AmazonReviewsClassification (fr) | accuracy | 33.450 |
| Classification | MTEB AmazonReviewsClassification (fr) | f1 | 33.080 |
| Classification | MTEB AmazonReviewsClassification (ja) | accuracy | 30.046 |
Using it via the API
Once AxForge deploys text2vec-multilingual for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (text2vec-multilingual below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"text2vec-multilingual","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.