Thanks for the PR! I don't have the bandwidth to test this personally at the moment, so will defer to AMD team. Also I do not have any RDNA GPUs on hand.

cc:@pnunna93

The docs for this PR livehere. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi@electron271
There's still a couple conflicts, mostly because we removed all of the imports related to IPEX. If you don't mind fixing those I think we can merge after that! Thanks!

will look through all this soon, sorry have been somewhat busy

Hi,
It looks like this breaks build compatibility for ROCm 6.1. I would be OK with dropping ROCm 6.1 compatibility if@pnunna93 agrees, but otherwise we would need to fix that build as well.

Apart from that, just a few linting issues to fix.

Hi, It looks like this breaks build compatibility for ROCm 6.1. I would be OK with dropping ROCm 6.1 compatibility if@pnunna93 agrees, but otherwise we would need to fix that build as well.

Apart from that, just a few linting issues to fix.

I agree, we can deprecate 6.1 compatibility

I've opened#1788 which removes the ROCm 6.1 build.

Did some regression testing compared to the main branch on W7900 (gfx1100), R9700 (gfx1201) and MI300x (gfx942) using the rocm/vllm:latest Docker image. There don't seem to be any regressions. Out of the 804 newly enabled tests on gfx1100 and gfx1201, 156 fail due to accuracy issues while the other 648 pass. Attaching some logs:

• W7900 (gfx1100)
  • PRbitsandbytes_tests_gfx1100.log
  • mainbitsandbytes_tests_gfx1100_main.log
• R9700 (gfx1201)
  • PRbitsandbytes_tests_gfx1201.log
  • mainbitsandbytes_tests_gfx1201_main.log
• MI300x (gfx942)
  • PRbitsandbytes_tests_gfx942.log
  • mainbitsandbytes_tests_gfx942_main.log

Thanks@sstamenk - that's quite useful! The failing tests seem to be mostly gemv with fp32. I think that's OK for now and can be addressed separately.

@electron271 If we fix the lint issues and merge conflict I'm happy to merge this in!