Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitbdd4170

Browse files
achartieryufeiwu-nv
authored andcommitted
[None][fix] Disable DeepGEMM for Qwen3 MoE Attention layers (NVIDIA#8087)
Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
1 parent76415e6 commitbdd4170

File tree

2 files changed

+3
-0
lines changed

2 files changed

+3
-0
lines changed

‎tensorrt_llm/_torch/models/modeling_qwen3.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ def __init__(
3434
fuse_qk_norm_rope:bool=True,
3535
attn_output_gate:bool=False,
3636
use_gemma_rms_norm:bool=False,
37+
disable_deep_gemm:bool=False,
3738
):
3839
config=model_config.pretrained_config
3940
self.pretrained_config=config
@@ -71,6 +72,7 @@ def __init__(
7172
config=model_config,
7273
attn_output_gate=self.attn_output_gate,
7374
use_gemma_rms_norm=use_gemma_rms_norm,
75+
disable_deep_gemm=disable_deep_gemm,
7476
)
7577

7678

‎tensorrt_llm/_torch/models/modeling_qwen3_moe.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ def __init__(self, model_config: ModelConfig[Qwen3MoeConfig],
167167
self.self_attn=Qwen3Attention(
168168
model_config,
169169
layer_idx=layer_idx,
170+
disable_deep_gemm=True,
170171
)
171172
self.mapping=model_config.mapping
172173
self.enable_attention_dp=self.mapping.enable_attention_dp

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp