- Notifications
You must be signed in to change notification settings - Fork14.2k
vulkan: Warptile tuning for Intel Xe2/Xe3#18178
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Conversation
| if ((device->vendor_id == VK_VENDOR_ID_INTEL) && (device->driver_id == vk::DriverId::eIntelProprietaryWindows)) { | ||
| if (device->coopmat_support && device->architecture == INTEL_XE2) { | ||
| // Xe2/Xe3 with coopmat enabled - warptile performance tuning | ||
| m_warptile = {512,128,128,16, subgroup_size_8,32,2, tm_m, tn_m, tk_m, subgroup_size_8 }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I wonder if this should actually be the large tile size?
Also, a quick google search suggests Xe2 has 64KB of register file per core, which with 512 invocations is only 32 registers each which seems very low. But I've never worked on this hardware so I'm just speculating.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hi@jeffbolznv Sure I can look at re-enabling large warptile size for Intelhere and then moving the warptile config from m_ to l_. I'll also check perf again after the change.
Are you doing (64 * 1024) / 512 invocations is 128 bytes per invocation and the assumption is 4 byte width register? (to get 32 registers per invocation?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes, that's the calculation I did.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks Jeff, forXe architecture each register in GRF was 32 Byte wide. But I need to look into the register situation a bit deeper
jeffbolznv commentedDec 18, 2025
CC@mmerecki in case this makes sense to also enable for Linux. |
ggml/src/ggml-vulkan/ggml-vulkan.cpp Outdated
| m_align =64; | ||
| s_align =32; | ||
| if ((device->vendor_id == VK_VENDOR_ID_INTEL) && (device->driver_id == vk::DriverId::eIntelProprietaryWindows)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ah it's good to see more tunes show up 😃. Please move your tune to line 2845 so that they're all placed together.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hi@netrunnereve thanks! I can do that but the wg_denoms change also needs to overwrite the default and be 'paired' with this tuning config to pass unit tests. Do you want me to make two separate 'if' statements with the same conditions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I would just move this section above line 2841.
l_mmq_wg_denoms = l_wg_denoms = {128, 128, 1 }; m_mmq_wg_denoms = m_wg_denoms = { 64, 64, 1 }; s_mmq_wg_denoms = s_wg_denoms = { 32, 32, 1 }; l_align = 128; m_align = 64; s_align = 32;There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Made the change
netrunnereve commentedDec 18, 2025
Oh and please run a before and after |
virajwad commentedDec 18, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Thanks! I checked on Arc B580, everything saw good improvement (except for type_a=bf16 test which was same perf)  My IGPU had same perf on all quants as it doesn't support coopmat |
Summary
In this PR we modify the m_warptile configurations to increase prompt processing performance for Intel Xe2+ GPUs that support coopmat extension
Changes
Accuracy Check
Basic testing w/ llama-cli across models that show perf jump (+ trying different prompt sizes) - model output looks correct / reasonable.
Unit tests Check
Checked on system w/ Arc B580 + Intel Integrated Graphics. All unit tests pass.
Performance Results
Command ran is
llama-bench.exe -m ..\<model> -p 512 -r 5 -n 128The eval token gen results don't change and weren't expected to, only prompt processing :) Below numbers show prompt processing in tok/s for Arc B580 and Lunar Lake Series 2 IGPU.
PR Status
Ready for Review