- Notifications
You must be signed in to change notification settings - Fork477
Vulkan: Select device with env variable, and skip initialize for unused devices.#629
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Conversation
wbruna commentedJun 19, 2025
Works for me... kind of 🙂 I don't think SD_VK_DEVICE can be areplacement for GGML_VK_VISIBLE_DEVICES: But it's still useful when used together: As I understand it, ggml excludes the iGPU by default, so it doesn't even show up on the device list unless it's explicitly enabled. (aside: I really wish there was a way to specify the devices by a stable ID. Notice how the '0' device points to different GPUs depending on the context...) |
stduhpf commentedJul 25, 2025
Ok I finally compiled sdcpp with Rocm, and at least it's selecting device 0 by default (and it's running faster than I thought compared to Vulkan), but It makes me think having an option to select backend device when initializing context could be a nice idea for all backends. I'm thining maybe passing It could also be worth it to find a way to select device for model parts, like end text encoders and VAE to device 1 and diffusion model to device 0 for example? |
Uh oh!
There was an error while loading.Please reload this page.
Introduces the
SD_VK_DEVICEenv variable that can be used to chose the device to run inference on.Also backend was initialized on every device for no apparent reason.
I know it was already possible to hide some devices from the Vulkan backend by setting the
GGML_VK_VISIBLE_DEVICESenv variable, but this isn't ideal, as it affects every program that uses Vulkan GGML.(Maybe the same should be done for CUDA and SYCL backends)