stable-diffusion.cpp

Features

TODO

Usage

Get the Code

git clone --recursive https://github.com/leejet/stable-diffusion.cppcd stable-diffusion.cpp

cd stable-diffusion.cppgit pull origin mastergit submodule initgit submodule update

Download weights

Build

Build from scratch

mkdir buildcd buildcmake ..cmake --build. --config Release

Using OpenBLAS

cmake .. -DGGML_OPENBLAS=ONcmake --build . --config Release

Using CUBLAS

cmake .. -DSD_CUBLAS=ONcmake --build . --config Release

Using HipBLAS

cmake .. -G "Ninja" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1100cmake --build . --config Release

Using Metal

cmake .. -DSD_METAL=ONcmake --build . --config Release

Using Flash Attention

cmake .. -DSD_FLASH_ATTN=ONcmake --build . --config Release

Run

usage: ./bin/sd [arguments]arguments:  -h, --help                         show this help message and exit  -M, --mode [MODEL]                 run mode (txt2img or img2img or convert, default: txt2img)  -t, --threads N                    number of threads to use during computation (default: -1).                                     If threads <= 0, then threads will be set to the number of CPU physical cores  -m, --model [MODEL]                path to model  --vae [VAE]                        path to vae  --taesd [TAESD_PATH]               path to taesd. Using Tiny AutoEncoder for fast decoding (low quality)  --control-net [CONTROL_PATH]       path to control net model  --embd-dir [EMBEDDING_PATH]        path to embeddings.  --stacked-id-embd-dir [DIR]        path to PHOTOMAKER stacked id embeddings.  --input-id-images-dir [DIR]        path to PHOTOMAKER input id images dir.  --normalize-input                  normalize PHOTOMAKER input id images  --upscale-model [ESRGAN_PATH]      path to esrgan model. Upscale images after generate, just RealESRGAN_x4plus_anime_6B supported by now.  --upscale-repeats                  Run the ESRGAN upscaler this many times (default 1)  --type [TYPE]                      weight type (f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0)                                     If not specified, the default is the type of the weight file.  --lora-model-dir [DIR]             lora model directory  -i, --init-img [IMAGE]             path to the input image, required by img2img  --control-image [IMAGE]            path to image condition, control net  -o, --output OUTPUT                path to write result image to (default: ./output.png)  -p, --prompt [PROMPT]              the prompt to render  -n, --negative-prompt PROMPT       the negative prompt (default: "")  --cfg-scale SCALE                  unconditional guidance scale: (default: 7.0)  --strength STRENGTH                strength for noising/unnoising (default: 0.75)  --style-ratio STYLE-RATIO          strength for keeping input identity (default: 20%)  --control-strength STRENGTH        strength to apply Control Net (default: 0.9)                                     1.0 corresponds to full destruction of information in init image  -H, --height H                     image height, in pixel space (default: 512)  -W, --width W                      image width, in pixel space (default: 512)  --sampling-method {euler, euler_a, heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm}                                     sampling method (default: "euler_a")  --steps  STEPS                     number of sample steps (default: 20)  --rng {std_default, cuda}          RNG (default: cuda)  -s SEED, --seed SEED               RNG seed (default: 42, use random seed for < 0)  -b, --batch-count COUNT            number of images to generate.  --schedule {discrete, karras, ays} Denoiser sigma schedule (default: discrete)  --clip-skip N                      ignore last layers of CLIP network; 1 ignores none, 2 ignores one layer (default: -1)                                     <= 0 represents unspecified, will be 1 for SD1.x, 2 for SD2.x  --vae-tiling                       process vae in tiles to reduce memory usage  --control-net-cpu                  keep controlnet in cpu (for low vram)  --canny                            apply canny preprocessor (edge detection)  --color                            colors the logging tags according to level  -v, --verbose                      print extra info

Quantization

Convert to GGUF

./bin/sd -M convert -m ../models/v1-5-pruned-emaonly.safetensors -o  ../models/v1-5-pruned-emaonly.q8_0.gguf -v --type q8_0

txt2img example

./bin/sd -m ../models/sd-v1-4.ckpt -p"a lovely cat"# ./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat"# ./bin/sd -m ../models/sd_xl_base_1.0.safetensors --vae ../models/sdxl_vae-fp16-fix.safetensors -H 1024 -W 1024 -p "a lovely cat" -v

img2img example

./bin/sd --mode img2img -m ../models/sd-v1-4.ckpt -p "cat with blue eyes" -i ./output.png -o ./img2img_output.png --strength 0.4

with LoRA

./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:marblesh:1>" --lora-model-dir ../models

LCM/LCM-LoRA

./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:lcm-lora-sdv1-5:1>" --steps 4 --lora-model-dir ../models -v --cfg-scale 1

Using TAESD to faster decoding

curl -L -O https://huggingface.co/madebyollin/taesd/blob/main/diffusion_pytorch_model.safetensors

sd -m ../models/v1-5-pruned-emaonly.safetensors -p"a lovely cat" --taesd ../models/diffusion_pytorch_model.safetensors

Using ESRGAN to upscale results

sd -m ../models/v1-5-pruned-emaonly.safetensors -p"a lovely cat" --upscale-model ../models/RealESRGAN_x4plus_anime_6B.pth

Using PhotoMaker to personalize image generation

bin/sd -m ../models/sdxlUnstableDiffusers_v11.safetensors  --vae ../models/sdxl_vae.safetensors --stacked-id-embd-dir ../models/photomaker-v1.safetensors --input-id-images-dir ../assets/examples/scarletthead_woman -p"a girl img, retro futurism, retro game art style but extremely beautiful, intricate details, masterpiece, best quality, space-themed, cosmic, celestial, stars, galaxies, nebulas, planets, science fiction, highly detailed" -n"realistic, photo-realistic, worst quality, greyscale, bad anatomy, bad hands, error, text" --cfg-scale 5.0  --sampling-method euler -H 1024 -W 1024 --style-ratio 10 --vae-on-cpu -o output.png

Docker

Building using Docker

docker build -t sd.

Run

docker run -v /path/to/models:/models -v /path/to/output/:/output sd [args...]# For example# docker run -v ./models:/models -v ./build:/output sd -m /models/sd-v1-4.ckpt -p "a lovely cat" -v -o /output/output.png

Memory Requirements

Bindings

UIs

Contributors

References