AxForge
Sign in Get started

Model reference · open weights

Llama-2

LLMs meta-llama Text gen 3 builds Open, with conditions 804k dl/mo

Llama-2 is an open-weight language model from Meta. Llama-2-7b-hf (FP16) weighs 13.5 GB; the smallest configuration that runs it is RTX 4060 Ti 16 GB.

  • Llama-2 is a 6.7B parameter generative text model developed by Meta for English text generation tasks.
  • It utilizes an optimized transformer architecture and is distributed under the llama2 license.
  • The model supports a context length of 4k tokens and is part of a family of models ranging from 7B to 70B parameters.

Summary of the meta-llama/Llama-2-7b-hf model card, 2026-10-01

Hugging Face ↗GitHub ↗Paper ↗

What it is

Released byMeta
Released2023-07-13
Parameters6.7B
VRAM13.5 GB for the weights

What it runs on

Memory and cards for Llama-2-7b-hf (FP16)

13.5 GBweights, file size
762 MBruntime overhead, at least

How much memory each request adds isn't estimated yet for this architecture. The weights need at least the cards below, plus room for the context.

CardWeights alone
RTX 3060 12 GBdoes not fit
RTX 4060 Ti 16 GBtight
RTX 3090 24 GBfits
RTX 4090 24 GBfits
RTX 5090 32 GBfits
L40S 48 GBfits
A100 80 GBfits
H100 80 GBfits
RTX PRO 6000 Blackwell 96 GBfits
DGX Spark (GB10) 128 GB unifiedfits
H200 141 GBfits
B200 180 GBfits

Builds

Sizes, precisions & builds

BuildParamsPrecisionWeightsSmallest setup
Llama-2-7b-hf (above) ↗ 6.7BFP16 13.5 GBRTX 4060 Ti 16 GB
Llama-2-13b-chat-hf ↗ 13.0BFP16 26.0 GBRTX 5090 32 GB
Llama-2-70b-chat-hf ↗ 69.0BFP16 138 GBB200 180 GB

How it works

How language models work

Your prompttext / messagesTransformerattention over tokensNext-token loopgenerate + streamResponsetext · tool callsA language model reads your tokens and predicts the next one, again and again, streaming the reply back.
© 2026 AxForge · EU-hosted AI infrastructure Pricing Docs Trust Privacy Terms