Model reference · open weights
indobert-lite-p1 is an open-weight embedding model from indobenchmark. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | indobenchmark |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Context | 512 tokens |
| Runs with | transformers |
| Released | 2022-03-02 |
| Popularity | 1k downloads / month |
| Licence | Open weights |
About
IndoBERT is a state-of-the-art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective.
| Model | #params | Arch. | Training data |
|---|---|---|---|
indobenchmark/indobert-base-p1 | 124.5M | Base | Indo4B (23.43 GB of text) |
indobenchmark/indobert-base-p2 | 124.5M | Base | Indo4B (23.43 GB of text) |
indobenchmark/indobert-large-p1 | 335.2M | Large | Indo4B (23.43 GB of text) |
indobenchmark/indobert-large-p2 | 335.2M | Large | Indo4B (23.43 GB of text) |
indobenchmark/indobert-lite-base-p1 | 11.7M | Base | Indo4B (23.43 GB of text) |
indobenchmark/indobert-lite-base-p2 | 11.7M | Base | Indo4B (23.43 GB of text) |
indobenchmark/indobert-lite-large-p1 | 17.7M | Large | Indo4B (23.43 GB of text) |
indobenchmark/indobert-lite-large-p2 | 17.7M | Large | Indo4B (23.43 GB of text) |
from transformers import BertTokenizer, AutoModel
tokenizer = BertTokenizer.from_pretrained("indobenchmark/indobert-lite-base-p1")
model = AutoModel.from_pretrained("indobenchmark/indobert-lite-base-p1")
x = torch.LongTensor(tokenizer.encode('aku adalah anak [MASK]')).view(1,-1)
print(x, model(x)[0].sum())
If you use our work, please cite:
@inproceedings{wilie2020indonlu,
title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},
author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},
booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},
year={2020}
}
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys indobert-lite-p1 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (indobert-lite-p1 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"indobert-lite-p1","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.