Model reference · open weights
HARC-Llama-3.1 is an open-weight language model from microsoft. HARC-Llama-3.1-8B-Instruct (BF16) weighs 16.1 GB; the smallest configuration that runs it is 2× RTX 3060 12 GB.
Summary of the microsoft/HARC-Llama-3.1-8B-Instruct model card, 2026-10-01
What it is
| Released by | microsoft |
|---|---|
| Released | 2026-07-02 |
| Parameters | 8.0B |
| VRAM | 16.1 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB … RTX 4060 Ti 16 GB 2 smaller cards | — | — | — | |
| RTX 3090 24 GB | 6 | 1 | 49K | 23.4 GB |
| RTX 4090 24 GB | 6 | 1 | 48K | 23.4 GB |
| RTX 5090 32 GB | 13 | 3 | 105K | 31.0 GB |
| L40S 48 GB | 25 | 6 | all 128K | 44.0 GB |
| A100 80 GB | 57 | 14 | all 128K | 78.2 GB |
| H100 80 GB | 53 | 13 | all 128K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 67 | 16 | all 128K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 80 | 20 | all 128K | 107 GB |
| H200 141 GB | 109 | 27 | all 128K | 138 GB |
| B200 180 GB | 143 | 35 | all 128K | 176 GB |
| 2× RTX 3060 12 GB tensor parallel | 5 | 1 | 42K | 11.6 GB a card |
| 2× RTX 4060 Ti 16 GB tensor parallel | 12 | 3 | 98K | 15.4 GB a card |
| 2× RTX 4090 24 GB tensor parallel | 27 | 6 | all 128K | 23.4 GB a card |
| 2× RTX 3090 24 GB tensor parallel | 27 | 6 | all 128K | 23.4 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 17.9 GB | 21.1 GB |
| 5 | 22.2 GB | 38.3 GB |
| 8 | 25.4 GB | 51.2 GB |
| 16 | 34.0 GB | 85.5 GB |
| 32 | 51.2 GB | 154 GB |
| 64 | 85.5 GB | 292 GB |
One card, with vLLM's small-card settings.
From the model card
HARC safety-alignment LoRA merged into meta-llama/Llama-3.1-8B-Instruct (full standalone model).
Part of the HARC release; see paper arXiv:2607.00572.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("microsoft/HARC-Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("microsoft/HARC-Llama-3.1-8B-Instruct")
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.