Model reference · open weights
flux-ip-adapter is an open-weight image model from XLabs-AI, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
<img src="https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/light/join-our-discord-rev1.png?raw=true" This repository provides a IP-Adapter checkpoint for FLUX.1-dev model by Black Forest Labs See our github for comfy ui workflows. Models IP-Adapter is trained on 512x512 resolution for 50k steps and 1024x1024 for 25k steps resolution and works for both 512x512 and 1024x1024 resolution. Model is training, we release new checkpoints regularly, stay updated. We release v1 version - which can be used directly in ComfyUI! Please, see our ComfyUI custom nodes installation guide Examples See examples of our models results below. Also, some generation results with input images are provided in "Files and versions" Inference To try our models, you have 2 options: 1. Use main.py from our official repo 2. Use our custom nodes for ComfyUI and test it with provided workflows (check out folder /workflows) Instruction for ComfyUI 1. Go to ComfyUI/customnodes 2. Clone x-flux-comfyui, path should be ComfyUI/customnodes/x-flux-comfyui/, where is all the files in this repo 3. Go to ComfyUI/customnodes/x-flux-comfyui/ and run python setup.py 4. Update x-flux-comfy with git pull or reinstall it. 5. Download Clip-L model.safetensors from OpenAI VIT CLIP large, and put it to ComfyUI/models/clipvision/. 6. Download our IPAdapter from huggingface, and put it to ComfyUI/models/xlabs/ipadapters/. 7. Use Flux Load IPAdapter and Apply Flux IPAdapter nodes, choose right CLIP model and enjoy your genereations. 8. You can find example workflow in folder workflows in this repo. If you get bad results, try to set truegs=2 Limitations The IP Adapter is currently in beta. We do not guarantee that you will get a good result right away, it may take more attempts to get a result. License Our weights fall under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | XLabs-AI |
|---|---|
| Type | Image models |
| Variants | 1 |
| Runs with | diffusers |
| Released | 2024-08-20 |
| Popularity | 40k downloads / month |
| Likes | 430 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| flux-ip-adapter | — | BF16 | — | — | Weights ↗ |
Using it via the API
Once AxForge deploys flux-ip-adapter for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (flux-ip-adapter below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/images/generations \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"flux-ip-adapter","prompt":"a red bicycle","size":"1024x1024"}'
Details
Languages
Trained / evaluated on
Tags
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗