Model reference · open weights

aya-vision

LLMs CohereForAI Vision + text 1 build Non-commercial 12k dl/mo

aya-vision is an open-weight language model from Cohere. aya-vision-32b (BF16) weighs 66.3 GB; the smallest configuration that runs it is H100 80 GB.

Aya Vision 8B is an open weights vision-language model developed by Cohere For AI for tasks such as OCR, captioning, and visual reasoning.
The model contains 8 billion parameters, supports a context length of 16K tokens, and is trained on 23 languages including English, French, and Japanese.
It is released under the CC-BY-NC license.

Summary of the unsloth/aya-vision-32b model card, 2026-10-01

What it is

What it runs on

Memory and cards for aya-vision-32b (BF16)

66.3 GBweights, file size

164 MBcache per 1K tokens

2.3 GBruntime overhead, at least

8,192 tokenscontext max

Card	Requests at once	Context max	Memory
Card	8K each	Context max	Memory
RTX 3060 12 GB … L40S 48 GB 6 smaller cards	—	—
A100 80 GB	7	all 8K	78.2 GB
H100 80 GB	2	all 8K	78.1 GB
RTX PRO 6000 Blackwell 96 GB	14	all 8K	93.8 GB
DGX Spark (GB10) 128 GB unified	24	all 8K	107 GB
H200 141 GB	47	all 8K	138 GB
B200 180 GB	74	all 8K	176 GB
2× L40S 48 GB tensor parallel	12	all 8K	44.0 GB a card
4× RTX 4090 24 GB tensor parallel	13	all 8K	23.4 GB a card
4× RTX 3090 24 GB tensor parallel	13	all 8K	23.4 GB a card
4× RTX 5090 32 GB tensor parallel	36	all 8K	31.0 GB a card
2× H100 80 GB tensor parallel	54	all 8K	78.1 GB a card
2× A100 80 GB tensor parallel	63	all 8K	78.2 GB a card

Memory needed at each load

One card, with vLLM's small-card settings.

Details

Languages

enfrdeesitptjakozharelfaplid