tensorrt_llm/lora_manager.py (1)

1031-1042:Make the swap robust: avoid mutating dict during iteration, assert even rows, and drop unnecessary clone.

• Mutatinglora_model[key] while iteratinglora_model.items() can be fragile. Iterate over a materialized list instead.
• Assert row count is even before splitting; otherwise a silent half-split can corrupt weights.
• The extra.clone() is not needed;torch.cat allocates a fresh tensor.

Apply this diff to harden the preprocessing:

-        def preprocess_lora_weights(lora_model, model_config):-            # Swap weights of gate_up_proj-            if getattr(model_config, "swap_gate_up_proj_lora_b_weight", True):-                for key, value in lora_model.items():-                    if "gate_up_proj.lora_B.weight" in key:-                        original_weights = value.contiguous().clone()-                        half_split = original_weights.shape[0] // 2-                        first_half = original_weights[:half_split, :]-                        second_half = original_weights[half_split:, :]-                        value = torch.cat((second_half, first_half), dim=0)-                        lora_model[key] = value-            return lora_model+        def preprocess_lora_weights(lora_model, model_config):+            # Swap rows for gate_up_proj.lora_B.weight (gate | up ordering)+            if getattr(model_config, "swap_gate_up_proj_lora_b_weight", True):+                for key, value in list(lora_model.items()):+                    if "gate_up_proj.lora_B.weight" in key:+                        w = value.contiguous()+                        rows = w.shape[0]+                        assert rows % 2 == 0, (+                            f"Expected even row count for {key}, got {rows}"+                        )+                        half = rows // 2+                        first_half = w[:half, :]+                        second_half = w[half:, :]+                        swapped = torch.cat((second_half, first_half), dim=0)+                        lora_model[key] = swapped+            return lora_model

tensorrt_llm/lora_helper.py (3)

1-15:Update SPDX year range to include 2025.

Repo guideline says to prepend NVIDIA copyright header (current year). Please bump to 2025.

Apply:

-# SPDX-FileCopyrightText: Copyright (c) 2022-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+# SPDX-FileCopyrightText: Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

---

22-41:Avoid duplicating “missing QKV” logic; reuse the manager’s single source of truth.

This function mirrorsLoraManager.get_missing_qkv_modules. Maintaining two implementations risks divergence.

Consider importing and delegating:

-from typing import Dict, List, Optional+from typing import Dict, List, Optional+from .lora_manager import LoraManager@@-def get_missing_qkv_modules_from_lora_modules(-        lora_target_modules: List[str]) -> List[str]:+def get_missing_qkv_modules_from_lora_modules(+        lora_target_modules: List[str]) -> List[str]:@@-    missing_qkv_modules = []-    if any(x in lora_target_modules for x in ["attn_q", "attn_k", "attn_v"]):-        for lora_module in ["attn_q", "attn_k", "attn_v"]:-            if lora_module not in lora_target_modules:-                missing_qkv_modules.append(lora_module)-    if any(x in lora_target_modules-           for x in ["cross_attn_q", "cross_attn_k", "cross_attn_v"]):-        for lora_module in ["cross_attn_q", "cross_attn_k", "cross_attn_v"]:-            if lora_module not in lora_target_modules:-                missing_qkv_modules.append(lora_module)-    return missing_qkv_modules+    return LoraManager.get_missing_qkv_modules(lora_target_modules)

---

43-59:De-duplicate default module mapping; import from a single location.

A second copy of the TRT-LLM↔HF mapping is hard to keep in sync (e.g., new modules likemlp_gate_up). Prefer re-exporting the mapping fromlora_manager.py.

For example:

-from ._utils import DictConversion+from ._utils import DictConversion+from .lora_manager import get_default_trtllm_modules_to_hf_modules as _default_map@@-def get_default_trtllm_modules_to_hf_modules():-    """Get default mapping from TensorRT-LLM module names to HuggingFace module names."""-    return {-        "attn_q": "q_proj",-        "attn_k": "k_proj",-        "attn_v": "v_proj",-        "attn_dense": "o_proj",-        "mlp_h_to_4h": "gate_proj",-        "mlp_4h_to_h": "down_proj",-        "mlp_gate": "up_proj",-        "mlp_gate_up": "gate_up_proj",-        "moe_h_to_4h": "w1",-        "moe_4h_to_h": "w2",-        "moe_gate": "w3",-        "moe_router": "gate",-    }+def get_default_trtllm_modules_to_hf_modules():+    """Get default mapping from TensorRT-LLM module names to HuggingFace module names."""+    return _default_map()

tensorrt_llm/_torch/models/modeling_phi4mm.py (1)

265-286:LoRA requests per modality — LGTM with a minor suggestion.

The static helper returns consistentLoRARequests for image/audio modalities. Minor nit: use_ for the throwaway loop variable.

Apply:

-                ) for i in range(num_requests)+                ) for _ in range(num_requests)

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

tensorrt_llm/lora_manager.py (2)

1053-1053:HF LoRA path preprocess hook — LGTM.

The guarded call topreprocess_lora_weights(lora_model, model_config) keeps HF-loading compatible across models while allowing Phi4MM to disable swapping.

---

244-244:swap_gate_up_proj_lora_b_weight flag fully propagated and verified
All instantiations, defaults, mappings, and tests correctly include or reference the newswap_gate_up_proj_lora_b_weight flag. No missing call sites or mappings were found.

tensorrt_llm/lora_helper.py (1)

61-80:LoRA dispatcher wrapper — LGTM.

The thin wrapper keeps call sites simple and preserves the existing public API.

tests/integration/defs/test_e2e.py (2)

2489-2491:Updated image expected keywords for Phi4MM — LGTM.

The new tokens reflect the revised LoRA targets/mappings and match the updated modality behavior.

---

2497-2498:Updated image_audio expected keywords for Phi4MM — LGTM.

Consistent with the modality handling changes.

tensorrt_llm/_torch/models/modeling_phi4mm.py (2)

259-261:Disable swap of gate_up_proj.lora_B.weight for Phi4MM — correct model-specific override.

Default is True globally; Phi4MM sets it to False here, which matches the intended accuracy fix.

---

246-257:✅ Confirmed mlp_gate_up registration and mapping
The LoRA target modules and mappings inmodeling_phi4mm.py correctly use"mlp_gate_up" →"gate_up_proj", and we’ve verified thatLORA_MODULE_IDS inlora_manager.py includes

• "mlp_gate_up": 18

No further changes needed.