Model reference · open weights
indoSBERT-large is an open-weight embedding model from denaya. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | denaya |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Context | 512 tokens |
| Runs with | sentence-transformers |
| Released | 2023-07-27 |
| Popularity | 2k downloads / month |
| Licence | Unknown |
About
This is a sentence-transformers model: It maps sentences & paragraphs to a 256 dimensional dense vector space and can be used for tasks like clustering or semantic search.
IndoSBERT is a modification of https://huggingface.co/indobenchmark/indobert-large-p1 that has been fine-tuned using the siamese network scheme inspired by SBERT (Reimers et al., 2019).
This model was fine-tuned with the STS Dataset (2012-2016) which was machine-translated into Indonesian languange.
This model can provide meaningful semantic sentence embeddings for Indonesian sentences.
Using this model becomes easy when you have sentence-transformers installed:
pip install -U sentence-transformers
Then you can use the model like this:
from sentence_transformers import SentenceTransformer
sentences = ["Komposer favorit saya adalah Joe Hisaishi", "Sapo tahu enak banget"]
model = SentenceTransformer('denaya/indoSBERT-large')
embeddings = model.encode(sentences)
print(embeddings)
The model was trained with the parameters:
DataLoader:
torch.utils.data.dataloader.DataLoader of length 1291 with parameters:
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
Loss:
sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss
Parameters of the fit()-Method:
{
"epochs": 50,
"evaluation_steps": 1,
"evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
"max_grad_norm": 1,
"optimizer_class": "",
"optimizer_params": {
"lr": 2e-05
},
"scheduler": "WarmupLinear",
"steps_per_epoch": null,
"warmup_steps": 100,
"weight_decay": 0.01
}
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
(2): Dense({'in_features': 1024, 'out_features': 256, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@article{author = {Diana, Denaya},
title = {IndoSBERT: Indonesian SBERT for Semantic Textual Similarity tasks},
year = {2023},
url = {https://huggingface.co/denaya/indoSBERT-large}
}
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys indosbert-large for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (indosbert-large below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"indosbert-large","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.