Model reference · open weights
llava-llama-3-v1_1-imat is an open-weight language model from city96. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | city96 |
|---|---|
| Type | Language models |
| Task | Vision + text |
| Runs with | gguf |
| Based on | xtuner/llava-llama-3-8b-v1_1-transformers |
| Released | 2024-12-20 |
| Popularity | 875 downloads / month |
| Licence | Unknown |
About
This is an imatrix gguf conversion of xtuner/llava-llama-3-8b-v1_1-transformers.
Mainly intended to be used as the text encoder for Hunyuan Video, but possible to use for vision tasks with the mmproj file from the xtuner gguf repository.
The imatrix dataset used was calibration_datav3.txt by Bartowski, which was used for all quants under Q6_K. Tested against wikitext / no imatrix and it outperformed both.
Note that the vocab_size is different between the transformers (128 320) and the hf (128 256) repositories. This used the former as it was what was used in the official Hunyuan Video code.
IQ quants will be slow in ComfyUI due to using numpy fallback.
From the published model card. Full card on the HuggingFace links in the sidebar.
How it works
Using it via the API
Once AxForge deploys llava-llama-3-v1-1-imat for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (llava-llama-3-v1-1-imat below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"llava-llama-3-v1-1-imat","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.