Model reference · open weights
ProteusV0.3 is an open-weight image model from dataautogpt3. ProteusV0.3 (BF16) weighs 6.9 GB; the smallest configuration that runs it is RTX 4060 Ti 16 GB.
What it is
| Released by | dataautogpt3 |
|---|---|
| Type | Image models |
| Task | Text→image |
| Runs with | diffusers |
| Released | 2024-02-12 |
| Popularity | 16k downloads / month |
| Weights | 6.9 GB (ProteusV0.3 (BF16), file size) |
| Licence | Open, with conditions |
What it runs on
Weights 6.9 GB (file size) · working memory for one 1024×1024 image about 5.0 GB · overhead about 537 MB.
| Card | One 1024×1024 image | Counted memory |
|---|---|---|
| RTX 3060 12 GB | does not fit | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
Estimates, not measurements: the weights are the build's file size; one 1024×1024 image needs about 5 GB of working memory (larger images more). diffusers can also place a pipeline's parts on separate cards (device_map) — not estimated here. Counted memory is 92 % of what CUDA reports for the card.
From the model card
Proteus V0.3 has been advanced with an additional 200,000 anime-related images, further refined by a selection of 15,000 aesthetically pleasing images, enhancing its lighting effects significantly. This upgrade preserves its understanding of prompts and maintains its photorealistic and stylistic capabilities without suffering from catastrophic forgetting.
Proteus serves as a sophisticated enhancement over OpenDalleV1.1, leveraging its core functionalities to deliver superior outcomes. Key areas of advancement include heightened responsiveness to prompts and augmented creative capacities. To achieve this, it was fine-tuned using approximately 220,000 GPTV captioned images from copyright-free stock images (with some anime included), which were then normalized. Additionally, DPO (Direct Preference Optimization) was employed through a collection of 10,000 carefully selected high-quality, AI-generated image pairs.
In pursuit of optimal performance, numerous LORA (Low-Rank Adaptation) models are trained independently before being selectively incorporated into the principal model via dynamic application methods. These techniques involve targeting particular segments within the model while avoiding interference with other areas during the learning phase. Consequently, Proteus exhibits marked improvements in portraying intricate facial characteristics and lifelike skin textures, all while sustaining commendable proficiency across various aesthetic domains, notably surrealism, anime, and cartoon-style visualizations.
Use these settings for the best results with ProteusV0.3:
CFG Scale: Use a CFG scale of 8 to 7
Steps: 20 to 60 steps for more detail, 20 steps for faster results.
Sampler: DPM++ 2M SDE
Scheduler: Karras
Resolution: 1280x1280 or 1024x1024
please also consider using these keep words to improve your prompts:
best quality, HD, ~*~aesthetic~*~.
if you are having trouble coming up with prompts you can use this GPT I put together to help you refine the prompt. https://chat.openai.com/g/g-RziQNoydR-diffusion-master
import torch
from diffusers import (
StableDiffusionXLPipeline,
KDPM2AncestralDiscreteScheduler,
AutoencoderKL
)
# Load VAE component
vae = AutoencoderKL.from_pretrained(
"madebyollin/sdxl-vae-fp16-fix",
torch_dtype=torch.float16
)
# Configure the pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
"dataautogpt3/ProteusV0.3",
vae=vae,
torch_dtype=torch.float16
)
pipe.scheduler = KDPM2AncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')
# Define prompts and generate image
prompt = "black fluffy gorgeous dangerous cat animal creature, large orange eyes, big fluffy ears, piercing gaze, full moon, dark ambiance, best quality, extremely detailed"
negative_prompt = "nsfw, bad quality, bad anatomy, worst quality, low quality, low resolutions, extra fingers, blur, blurry, ugly, wrongs proportions, watermark, image artifacts, lowres, ugly, jpeg artifacts, deformed, noisy image"
image = pipe(
prompt,
negative_prompt=negative_prompt,
width=1024,
height=1024,
guidance_scale=7,
num_inference_steps=20
).images[0]
please support the work I do through donating to me on: https://www.buymeacoffee.com/DataVoid or following me on https://twitter.com/DataPlusEngine
Quoted from the model card on Hugging Face — the full card is behind the Hugging Face link above.
Running it yourself
Rent a machine by the hour — ComfyUI is installed on it. Open ComfyUI through the tunnel: its default workflow loads a checkpoint — choose this model's file in Load Checkpoint, with the settings its model card gives.
# on your rented machine: pip install diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained("dataautogpt3/ProteusV0.3", torch_dtype=torch.bfloat16).to("cuda")
image = pipe(prompt="a red bicycle on a cobbled street").images[0]
image.save("/workspace/out.png")# on your rented machine (the ssh line is on its page in the console)
# get REPO FILE FOLDER: one file into /workspace/models/FOLDER, where ComfyUI loads it from
get() { hf download "$1" "$2" --local-dir /workspace/hf-files && mkdir -p "/workspace/models/$3" && mv "/workspace/hf-files/$2" "/workspace/models/$3/$4"; }
# the model (6.5 GB)
get dataautogpt3/ProteusV0.3 ProteusV0.3.safetensors checkpoints
start-comfyui# on your computer, in a second terminal: ComfyUI in your browser at http://localhost:8188
# HOST and PORT are your machine's, from its page in the console
ssh -L 8188:localhost:8188 dev@HOST -p PORT