Model reference · open weights
llava-llama-3-v1_1-imat is an open-weight language model from city96. llava-llama-3-8b-v1_1-imat-gguf (GGUF) weighs 4.9 GB; the smallest configuration that runs it is RTX 3060 12 GB.
Summary of the city96/llava-llama-3-8b-v1_1-imat-gguf model card, 2026-10-01
What it is
| Released by | city96 |
|---|---|
| Released | 2024-12-20 |
| VRAM | 4.9 GB for the weights |
What it runs on
How much memory each request adds isn't estimated yet for this architecture. The weights need at least the cards below, plus room for the context.
| Card | Weights alone |
|---|---|
| RTX 3060 12 GB | fits |
| RTX 4060 Ti 16 GB | fits |
| RTX 3090 24 GB | fits |
| RTX 4090 24 GB | fits |
| RTX 5090 32 GB | fits |
| L40S 48 GB | fits |
| A100 80 GB | fits |
| H100 80 GB | fits |
| RTX PRO 6000 Blackwell 96 GB | fits |
| DGX Spark (GB10) 128 GB unified | fits |
| H200 141 GB | fits |
| B200 180 GB | fits |
From the model card
This is an imatrix gguf conversion of xtuner/llava-llama-3-8b-v1_1-transformers.
Mainly intended to be used as the text encoder for Hunyuan Video, but possible to use for vision tasks with the mmproj file from the xtuner gguf repository.
The imatrix dataset used was calibration_datav3.txt by Bartowski, which was used for all quants under Q6_K. Tested against wikitext / no imatrix and it outperformed both.
Note that the vocab_size is different between the transformers (128 320) and the hf (128 256) repositories. This used the former as it was what was used in the official Hunyuan Video code.
IQ quants will be slow in ComfyUI due to using numpy fallback.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
How it works