Model reference · open weights

gemma

LLMs google Text gen 2 builds Open, with conditions 134k dl/mo

gemma is an open-weight language model from Google. gemma-7b (BF16) weighs 34.2 GB; the smallest configuration that runs it is L40S 48 GB.

Gemma is a lightweight, open-weight text generation model developed by Google, designed for tasks such as question answering, summarization, and reasoning.
This 2.5B parameter version is a decoder-only large language model trained on 6 trillion tokens with an 8192 token context length.
It is available under the Gemma license and supports deployment on limited resources like laptops or desktops.

Summary of the google/gemma-2b model card, 2026-10-01. The estimate below is for another build of the family.

What it is

What it runs on

Memory and cards for gemma-7b (BF16)

34.2 GBweights, file size

651 MBruntime overhead, at least

How much memory each request adds isn't estimated yet for this architecture. The weights need at least the cards below, plus room for the context.

Card	Weights alone
RTX 3060 12 GB … RTX 5090 32 GB 5 smaller cards	does not fit
L40S 48 GB	fits
A100 80 GB	fits
H100 80 GB	fits
RTX PRO 6000 Blackwell 96 GB	fits
DGX Spark (GB10) 128 GB unified	fits
H200 141 GB	fits
B200 180 GB	fits
2× RTX 4090 24 GB split by layers (llama.cpp)	fits
2× RTX 3090 24 GB split by layers (llama.cpp)	fits
2× RTX 5090 32 GB split by layers (llama.cpp)	fits

Builds

Build	Params	Precision	Weights	Smallest setup
gemma-2b ↗	2.5B	BF16	10.0 GB	RTX 3060 12 GB
gemma-7b (above) ↗	8.5B	BF16	34.2 GB	L40S 48 GB

How it works

Details