Model reference · open weights
BERTA is an open-weight embedding model from sergeyzh. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | sergeyzh |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Parameters (lead) | 128M |
| Context | 512 tokens |
| Runs with | sentence-transformers |
| Based on | sergeyzh/LaBSE-ru-turbo |
| Released | 2025-03-10 |
| Popularity | 18k downloads / month |
| Licence | Open weights |
About
Модель для расчетов эмбеддингов предложений на русском и английском языках получена методом дистилляции эмбеддингов ai-forever/FRIDA (размер эмбеддингов - 1536, слоёв - 24) в sergeyzh/LaBSE-ru-turbo (размер эмбеддингов - 768, слоёв - 12). Основной режим использования FRIDA - CLS pooling заменен на mean pooling. Каких-либо других изменений поведения модели не производилось. Дистиляция выполнена в максимально возможном объеме - эмбеддинги русских и английских предложений, работа префиксов.
Размер контекста модели соответствует FRIDA - 512 токенов.
Все префиксы унаследованы от FRIDA. Оптимальный (обеспечивающий средние результаты) префикс для большинства задач - "categorize_entailment: " прописан по умолчанию в config_sentence_transformers.json
Перечень используемых префиксов и их влияние на оценки модели в encodechka:
| Префикс | STS | PI | NLI | SA | TI |
|---|---|---|---|---|---|
| - | 0,842 | 0,757 | 0,463 | 0,830 | 0,985 |
| search_query: | 0,853 | 0,767 | 0,479 | 0,825 | 0,987 |
| search_document: | 0,831 | 0,749 | 0,463 | 0,817 | 0,986 |
| paraphrase: | 0,847 | 0,778 | 0,446 | 0,825 | 0,986 |
| categorize: | 0,857 | 0,765 | 0,501 | 0,829 | 0,988 |
| categorize_sentiment: | 0,589 | 0,535 | 0,417 | 0,805 | 0,982 |
| categorize_topic: | 0,740 | 0,521 | 0,396 | 0,770 | 0,982 |
| categorize_entailment: | 0,841 | 0,762 | 0,571 | 0,827 | 0,986 |
Задачи:
Оценки модели на бенчмарке ruMTEB:
| Model Name | Metric | FRIDA | BERTA | rubert-mini-frida | multilingual-e5-large-instruct | multilingual-e5-large |
|---|---|---|---|---|---|---|
| CEDRClassification | Accuracy | 0.646 | 0.622 | 0.552 | 0.500 | 0.448 |
| GeoreviewClassification | Accuracy | 0.577 | 0.548 | 0.464 | 0.559 | 0.497 |
| GeoreviewClusteringP2P | V-measure | 0.783 | 0.738 | 0.698 | 0.743 | 0.605 |
| HeadlineClassification | Accuracy | 0.890 | 0.891 | 0.880 | 0.862 | 0.758 |
| InappropriatenessClassification | Accuracy | 0.783 | 0.748 | 0.698 | 0.655 | 0.616 |
| KinopoiskClassification | Accuracy | 0.705 | 0.678 | 0.595 | 0.661 | 0.566 |
| RiaNewsRetrieval | NDCG@10 | 0.868 | 0.816 | 0.721 | 0.824 | 0.807 |
| RuBQReranking | MAP@10 | 0.771 | 0.752 | 0.711 | 0.717 | 0.756 |
| RuBQRetrieval | NDCG@10 | 0.724 | 0.710 | 0.654 | 0.692 | 0.741 |
| RuReviewsClassification | Accuracy | 0.751 | 0.723 | 0.658 | 0.686 | 0.653 |
| RuSTSBenchmarkSTS | Pearson correlation | 0.814 | 0.822 | 0.803 | 0.840 | 0.831 |
| RuSciBenchGRNTIClassification | Accuracy | 0.699 | 0.690 | 0.625 | 0.651 | 0.582 |
| RuSciBenchGRNTIClusteringP2P | V-measure | 0.670 | 0.650 | 0.586 | 0.622 | 0.520 |
| RuSciBenchOECDClassification | Accuracy | 0.546 | 0.555 | 0.493 | 0.502 | 0.445 |
| RuSciBenchOECDClusteringP2P | V-measure | 0.566 | 0.556 | 0.507 | 0.528 | 0.450 |
| SensitiveTopicsClassification | Accuracy | 0.398 | 0.399 | 0.373 | 0.323 | 0.257 |
| TERRaClassification | Average Precision | 0.665 | 0.657 | 0.606 | 0.639 | 0.584 |
| Model Name | Metric | FRIDA | BERTA | rubert-mini-frida | multilingual-e5-large-instruct | multilingual-e5-large |
|---|---|---|---|---|---|---|
| Classification | Accuracy | 0.707 | 0.698 | 0.631 | 0.654 | 0.588 |
| Clustering | V-measure | 0.673 | 0.648 | 0.597 | 0.631 | 0.525 |
| MultiLabelClassification | Accuracy | 0.522 | 0.510 | 0.463 | 0.412 | 0.353 |
| PairClassification | Average Precision | 0.665 | 0.657 |
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys berta for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (berta below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"berta","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.