Model reference · open weights
stable-diffusion-inpainting is an open-weight image model from stable-diffusion-v1-5, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Stable Diffusion Inpainting model card ⚠️ This repository is a mirror of the now deprecated ruwnayml/stable-diffusion-inpainting, this repository or oganization are not affiliated in any way with RunwayML. Modifications to the original model card are in <span style="color:crimson"red</span or <span style="color:darkgreen"green</span Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask. The Stable-Diffusion-Inpainting was initialized with the weights of the Stable-Diffusion-v-1-2. First 595k steps regular training, then 440k steps of inpainting training at resolution 512x512 on “laion-aesthetics v2 5+” and 10% dropping of the text-conditioning to improve classifier-free classifier-free guidance sampling. For inpainting, the UNet has 5 additional input channels (4 for the encoded masked-image and 1 for the mask itself) whose weights were zero-initialized after restoring the non-inpainting checkpoint. During training, we generate synthetic masks and in 25% mask everything. Open In Spaces | [](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/inpaintingwithstablediffusionusingdiffusers.ipynb) :-------------------------:|:-------------------------:| Examples: You can use this both with the 🧨Diffusers library and RunwayML GitHub repository (<span style="color:crimson"now deprecated</span), <span style="color:darkgreen"Automatic1111</span. Use with Diffusers How it works: image | maskimage :-------------------------:|:-------------------------:| prompt | Output :-------------------------:|:-------------------------:| Use with Original GitHub Repository <span style="color:darkgreen"or AUTOMATIC1111</span 1. Download the weights sd-v1-5-inpainting.ckpt 2. Follow instructions here (<span style="color:crimson"now deprecated</span). 3. <span style="color:darkgreen"Use it with <a href="https://github.com/AUTOMATIC1111/stable-diffusion-webui"AUTOMATIC1111</a</span Model Details - Developed by: Robin Rombach, Patrick Esser - Model type: Diffusion-based text-to-image generation model - Language(
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | stable-diffusion-v1-5 |
|---|---|
| Type | Image models |
| Variants | 1 |
| Runs with | diffusers |
| Released | 2024-08-30 |
| Popularity | 156k downloads / month |
| Likes | 118 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| stable-diffusion-inpainting | — | BF16 | — | — | Weights ↗ |
Using it via the API
Once AxForge deploys stable-diffusion-inpainting for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (stable-diffusion-inpainting below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/images/generations \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"stable-diffusion-inpainting","prompt":"a red bicycle","size":"1024x1024"}'
Details
Tags
Papers
Licence
Open weights under creativeml-openrail-m — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗