Model reference · open weights
llava-1.5 is an open-weight language model from llava-hf, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
LLaVA Model Card Below is the model card of Llava model 7b, which is copied from the original Llava model card that you can find here. Check out also the Google Colab demo to run Llava on a free-tier Google Colab instance: [](https://colab.research.google.com/drive/1qsl6cd2c8gGtEW1xV5io7S8NHh-Cp1TV?usp=sharing) Or check out our Spaces demo! [](https://huggingface.co/spaces/llava-hf/llava-4bit) Model details Model type: LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. Model date: LLaVA-v1.5-7B was trained in September 2023. Paper or resources for more information: https://llava-vl.github.io/ How to use the model First, make sure to have transformers = 4.35.3. The model supports multi-image and multi-prompt generation. Meaning that you can pass multiple images in your prompt. Make sure also to follow the correct prompt template (USER: xxx\nASSISTANT:) and add the token <image to the location where you want to query images: Using pipeline: Below we used "llava-hf/llava-1.5-7b-hf" checkpoint. Using pure transformers: Below is an example script to run generation in float16 precision on a GPU device: From transformers=v4.48, you can also pass image url or local path to the conversation history, and let the chat template handle the rest. Chat template will load the image for you and return inputs in torch.Tensor which you can pass directly to model.generate() Model optimization 4-bit quantization through bitsandbytes library First make sure to install bitsandbytes, pip install bitsandbytes and make sure to have access to a CUDA compatible GPU device. Simply change the snippet above with: Use Flash-Attention 2 to further speed-up generation First make sure to install flash-attn. Refer to the original repository of Flash Attention regarding that package installation. Simply change the snippet above with: License Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | llava-hf |
|---|---|
| Type | Language models |
| Parameters (lead) | 7.1B |
| Variants | 1 |
| Runs with | transformers |
| Released | 2023-12-05 |
| Popularity | 2.1M downloads / month |
| Likes | 372 |
| Licence | Open, with conditions |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| llava-1.5-7b-hf | 7.1B | BF16 | ~16.2 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys llava-1-5 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (llava-1-5 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"llava-1-5","messages":[{"role":"user","content":"Hello"}]}'
Details
Languages
Trained / evaluated on
Tags
Licence
Open weights under llama2, which carries conditions (e.g. attribution or an acceptable-use clause). Worth a read before production use — we can walk you through it. Read the licence ↗