Model reference · open weights
ARC4_Encoder_Llama is an open-weight embedding model from kyutai, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
ARC-Encoder models This page houses ARC4-EncoderLlama from four different versions of pretrained ARC-Encoders. Architectures and methods to train them are described in the paper ARC-Encoder: learning compressed text representations for large language models available here. Code: ARC-Encoder repository Models Details All the encoders released here are trained on web crawl filtered using Dactory based on a Llama3.2-3B base backbone. It consists in two ARC-Encoder specifically trained for one decoder and one for two decoders in the same time: - ARC8-EncoderLlama, trained on 2.6B tokens on Llama3.1-8B base specifically with a pooling factor of 8. - ARC8-EncoderMistral, trained on 2.6B tokens on Mistral-7B base specifically with a pooling factor of 8. - ARC8-Encodermulti, trained by sampling among the two decoders with a pooling factor of 8. - ARC4-EncoderLlama, trained on 2.6B tokens on Llama3.1-8B base specifically with a pooling factor of 4. Uses As described in the paper, the pretrained ARC-Encoders can be fine-tuned to perform various downstream tasks. You can also adapt an ARC-Encoder to a new pooling factor (PF) by fine-tuning it on the desired PF. For optimal results, we recommend fine-tuning toward a lower PF than the one used during pretraining. To reproduce the results presented in the paper, you can use our released fine-tuning dataset, ARCfinetuning. Licensing ARC-Encoders are licensed under the CC-BY 4.0 license. Terms of use: As the released models are pretrained from Llama3.2 3B backbone, ARC-Encoders are subject to the Llama Terms of Use found at Llama license. Citations If you use one of these models, please cite:
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | kyutai |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 3.0B |
| Variants | 1 |
| Released | 2026-03-26 |
| Likes | 2 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| ARC4_Encoder_Llama | 3.0B | BF16 | ~7 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys arc4-encoder-llama for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (arc4-encoder-llama below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"arc4-encoder-llama","input":"text to embed"}'
Licence
Open weights under cc-by-4.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗
Explore