Model reference · open weights
ko-sroberta-multitask is an open-weight embedding model from jhgan, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
ko-sroberta-multitask This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Usage (Sentence-Transformers) Using this model becomes easy when you have sentence-transformers installed: Then you can use the model like this: Usage (HuggingFace Transformers) Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings. Evaluation Results KorSTS, KorNLI 학습 데이터셋으로 멀티 태스크 학습을 진행한 후 KorSTS 평가 데이터셋으로 평가한 결과입니다. - Cosine Pearson: 84.77 - Cosine Spearman: 85.60 - Euclidean Pearson: 83.71 - Euclidean Spearman: 84.40 - Manhattan Pearson: 83.70 - Manhattan Spearman: 84.38 - Dot Pearson: 82.42 - Dot Spearman: 82.33 Training The model was trained with the parameters: DataLoader: sentencetransformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader of length 8885 with parameters: Loss: sentencetransformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss with parameters: DataLoader: torch.utils.data.dataloader.DataLoader of length 719 with parameters: Loss: sentencetransformers.losses.CosineSimilarityLoss.CosineSimilarityLoss Parameters of the fit()-Method: Full Model Architecture Citing & Authors - Ham, J., Choe, Y. J., Park, K., Choi, I., & Soh, H. (2020). Kornli and korsts: New benchmark datasets for korean natural language understanding. arXiv preprint arXiv:2004.03289 - Reimers, Nils and Iryna Gurevych. “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.” ArXiv abs/1908.10084 (2019) - Reimers, Nils and Iryna Gurevych. “Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation.” EMNLP (2020).
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | jhgan |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 111M |
| Context | 514 tokens |
| Variants | 1 |
| Runs with | sentence-transformers |
| Released | 2022-03-02 |
| Popularity | 926k downloads / month |
| Likes | 150 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| ko-sroberta-multitask | 111M | BF16 | ~0.3 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys ko-sroberta-multitask for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (ko-sroberta-multitask below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"ko-sroberta-multitask","input":"text to embed"}'
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗
Explore