- Notifications
You must be signed in to change notification settings - Fork1.9k
[TRTLLM-4629] [feat] Add support of CUDA13 and sm103 devices#7568
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes from1 commit
Commits
Show all changes
108 commits Select commitHold shift + click to select a range
303604f upgrade to base image and new TRT, fix many dependency issues
VALLIS-NERIA5c09dc8 CUDA13 breaking changes: c++ compile successful
VALLIS-NERIA1b84604 fix kernel select code to recognize sm103/sm100f
VALLIS-NERIA3a94d80 Update SM100f cubins
Tom-Zheng469a38d feat: Add support for SM103 3xFP4 tile shapes
djns9952ad443 disable 3xfp4
VALLIS-NERIA345c2bc update trtllm-gen sm100f cubins of gemm kernels
VALLIS-NERIAe27cbb5 Ampere moe kernel should build to all arch
VALLIS-NERIA78a55b8 fix vicuna dependency
VALLIS-NERIA271916d fix deep_gemm & CUDA13
VALLIS-NERIA886437d merge existing env fix
VALLIS-NERIAb782b6e fix sm check of kv reuse and chunked context
VALLIS-NERIA84f96b4 update triton and fix deepgemm pip
VALLIS-NERIA759e7a0 Merge remote-tracking branch 'gitlab/main' into feat/gb110_bringup
VALLIS-NERIAbee1df9 remove deepgemm war
VALLIS-NERIA97a3788 update triton image
VALLIS-NERIAebec4ea infra: upgrade to DLFW 25.08-pre and TRT 10.13.2.4
ZhanruiSunCh36f2e88 Merge branch 'user/zhanruis/update_dlfw_and_cu13' into 'feat/b300_cu13'
ZhanruiSunCh0bf6a18 Fix and waive to clean L0
VALLIS-NERIAf12a90b Merge branch 'feat/gb110_bringup' into 'feat/b300_cu13'
VALLIS-NERIA8c99853 infra: Support build for both CU12 and CU13
ZhanruiSunChc1014e8 Merge branch 'user/zhanruis/update_dlfw_and_cu13_2' into 'feat/b300_c…
ZhanruiSunCh4a95d88 revert tlg kernels for ease of merge
VALLIS-NERIA8b53236 Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_mai…
VALLIS-NERIA5391191 update tg cubins (temp ver)
VALLIS-NERIAf4de884 Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_mai…
VALLIS-NERIAb7cc06c disable merge waive list stage
VALLIS-NERIAfa8b52e fix more sm version check
VALLIS-NERIA808059d Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_mai…
VALLIS-NERIA90a9bc4 fix build error
VALLIS-NERIA80ea062 fix cubins
VALLIS-NERIA66b1d8d Update flashinfer
VALLIS-NERIAab7febd Merge commit '31979aefacbf80d2742c98ef30385db162788c84' into feat/b30…
VALLIS-NERIAb1c6f6a update cutlass and DeepGEMM
VALLIS-NERIA9ad68de Merge branch 'user/xiweny/update_cutlass_4.2' into 'feat/b300_cu13'
VALLIS-NERIAee37589 infra: update DLFW 25.08 GA, triton 25.08 GA
ZhanruiSunChc2e1ad9 Merge branch 'user/zhanruis/update_dlfw_and_cu13_3' into 'feat/b300_c…
ZhanruiSunCh6fd765f [None][fix] fix trtllm moe backend error when running gptoss on b300
jiagancf14c740 Merge branch 'dev-jiaganc-fix-b300-gptoss-trtllm' into 'feat/b300_cu13'
VALLIS-NERIA3c06303 [TRTLLM-7755][infra] Add DGX_B300 and GB300 tests in CI
yiqingy0c425c12 Merge branch 'user/yiqingy/add_b300_tests' into 'feat/b300_cu13'
yiqingy00fb835d fix cutlass moe not falling back
VALLIS-NERIA8d5a7ea [https://nvbugs/5443053][fix] Disable finalize fusion when Lora is used
jiaganc3cc2591 Merge branch 'dev-jiaganc-fix-b300-moe-lora' into 'feat/b300_cu13'
VALLIS-NERIA3805f61 [https://nvbugs/5453949][infra] unwaive test_llama_eagle3
bo-nva765ee4 Merge branch 'feat/b300_cu13-latest' into 'feat/b300_cu13'
VALLIS-NERIA14154ec disable sm103 moe kernel
VALLIS-NERIA38ef850 Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_0901
VALLIS-NERIA62a7897 Merge remote-tracking branch 'origin/main' into user/xiweny/merge_0901
VALLIS-NERIA90ce786 Fix arg name in _test_trtllm_serve_multimodal_benchmark.py
VALLIS-NERIA5bd50d4 update mha cubins and support 103a
VALLIS-NERIA1978227 Merge branch 'user/xiweny/mha_103' into 'feat/b300_cu13'
VALLIS-NERIA5ca3376 Support DLFW sanity check use CU13 image
ZhanruiSunCh9ae01a8 Merge branch 'user/zhanruis/0828_support_cuda_13_for_sanity_check' in…
ZhanruiSunCh973fd37 add 3xfp4 cutlass gemm
VALLIS-NERIAfcf413e Merge branch 'user/xiweny/3xfp4_gemm' into 'feat/b300_cu13'
VALLIS-NERIA5d4f7f4 update flashinfer and waive bug
VALLIS-NERIA22219bc Add B300 & GB300 CI
VALLIS-NERIA2c3f4cb Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAf8864b9 update trtllm gemm
VALLIS-NERIAcca347e [TRTLLM-4629] [feat] Step1: trtllm-gen kernels support sm103
VALLIS-NERIA5e7aa76 Merge branch 'user/sm103_trtllmgen' into feat/b300_cu13
VALLIS-NERIA10af4f4 [TRTLLM-4629] [feat] Step1: trtllm-gen kernels support sm103
VALLIS-NERIA1d7979a fix
VALLIS-NERIA3e71ec7 Merge branch 'user/sm103_trtllmgen' into feat/b300_cu13
VALLIS-NERIA65f8478 fix trtllm-gen interface change
VALLIS-NERIAbec1e71 fix
VALLIS-NERIA0b0781f fix
VALLIS-NERIA3d4f49e fix missing gemm kernels
VALLIS-NERIA1150def Merge branch 'user/sm103_trtllmgen' into feat/b300_cu13
VALLIS-NERIAd12eb4b fix CI build archs
VALLIS-NERIA322db71 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIA8f8766a waive
VALLIS-NERIA2912908 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAe6bb1fe remove non-exist cases
VALLIS-NERIA77657de fix build args
VALLIS-NERIAd42201e remove waivers and cleanup
VALLIS-NERIAcaea58a increase build memory
VALLIS-NERIAd4d9e77 reset build memory
VALLIS-NERIA019b1db fix 5505835
VALLIS-NERIAfdaf4e2 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAe30e0c8 waive
VALLIS-NERIA4cf9fed Merge commit 'ed27a72bcf71f7ab0e7137f7999988c9de82386f' into feat/b30…
VALLIS-NERIAb573e07 [None][infra] Disable CU12 build to save build time (cost > 5 hours o…
ZhanruiSunCh82833fa address comments
VALLIS-NERIA8cc5ea3 add comment
VALLIS-NERIAa8b630f Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIA2c287d5 don't throw in ctor
VALLIS-NERIA11d603b fix
VALLIS-NERIAd16d98c fix missing change
VALLIS-NERIA5f508b7 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIA2e61526 fix
VALLIS-NERIA0b73a57 refine sm version check
VALLIS-NERIA27c73de add a line of comment
VALLIS-NERIAb8d1ee6 exclude sm70
VALLIS-NERIA6133354 fix sm check
VALLIS-NERIA41d3cf6 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAced6e74 [None][infra] Remove WAR on feat branch (#7642)
ZhanruiSunCh98cbab0 [None][infra] Update images (#7690)
ZhanruiSunCh514ebc2 remove sm70 from fmha_v2 completely
VALLIS-NERIA9bd8df7 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAad20048 remove sm72 & 75
VALLIS-NERIA93195ec waive
VALLIS-NERIA98d42f9 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAcf74f40 fix testdb
VALLIS-NERIAd48e82a fix testdb
VALLIS-NERIA7657d83 fix
VALLIS-NERIA0192299 Merge remote-tracking branch 'origin/main' into feat/b300_cu13
VALLIS-NERIAFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
fix
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
- Loading branch information
Uh oh!
There was an error while loading.Please reload this page.
commit7657d83553ad27ed6e686d6d5d94889b51b8f71e
There are no files selected for viewing
2 changes: 0 additions & 2 deletionstests/unittest/llmapi/test_llm.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.