tensorrt_llm/_torch/speculative/eagle3.py (1)

17-19:TYPE_CHECKING import: LGTM

Static-only import avoids runtime dependency cycles. Optionally enable postponed annotations to drop quotes.

Apply this small tweak (optional):

+from __future__ import annotations

tests/unittest/llmapi/test_llm_pytorch.py (1)

229-233:Broaden coverage: parametrize enable_logprobs for both True/False

Current test only covers logprobs=True. Include False to exercise the non-logprobs path with embedding_bias.

Apply:

-@pytest.mark.parametrize(-    "enable_logprobs",-    [True],-)+@pytest.mark.parametrize("enable_logprobs", [False, True])

Also applies to: 253-259

tensorrt_llm/_torch/pyexecutor/sampler.py (4)

166-192:Consider extracting magic values as constants

The batched sampling functions use hardcoded default values. Consider extracting them for maintainability.

+# Sampling defaults+DEFAULT_TOP_K = 50+DEFAULT_TOP_P = 0.9+DEFAULT_TEMPERATURE = 1.0+MIN_TEMPERATURE = 1e-5+ def top_k_sampling_batch(     logits,-    top_k=50,+    top_k=DEFAULT_TOP_K,     generator: Optional[torch.Generator] = None ) -> tuple[torch.Tensor, torch.Tensor]:

 def top_p_sampling_batch(     logits: torch.Tensor,-    top_p: float = 0.9,-    temperature: float = 1.0,+    top_p: float = DEFAULT_TOP_P,+    temperature: float = DEFAULT_TEMPERATURE,     generator: Optional[torch.Generator] = None ) -> tuple[torch.Tensor, torch.Tensor]:

---

353-361:Add type hints for improved maintainability

The function would benefit from explicit return type hints.

 def _group_requests_by_sampling_strategy(-        requests: Iterable[LlmRequest]) -> dict[Strategy, torch.IntTensor]:+        requests: Iterable[LlmRequest]) -> dict[Strategy, torch.IntTensor]:     strategy_dict: dict[Strategy, list[int]] = defaultdict(list)

---

1052-1190:Well-structured batched sampling pipeline

The_sample_batched_by_strategy method effectively groups requests by strategy and processes them in batches. The handling of draft logits indices for speculation is properly tracked.

One concern: The d2t validation at lines 1071-1075 might be better placed inModelDrafter.prepare_draft_tokens as suggested by the FIXME comment.

Consider moving the d2t validation to ModelDrafter as suggested by the FIXME comment to maintain better separation of concerns.

---

1332-1333:Fix line length violations

These lines exceed the 120-character limit.

-            #    range(next_context_req_offsets_cuda[i] - req_num_steps_fictitious_cuda[i], next_context_req_offsets_cuda[i])-            # And if generation requests are present, those tensors include a trailing entries selecting all tokens/logits+            #    range(next_context_req_offsets_cuda[i] - req_num_steps_fictitious_cuda[i],+            #          next_context_req_offsets_cuda[i])+            # And if generation requests are present, those tensors include a trailing+            # entries selecting all tokens/logits

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

tensorrt_llm/_torch/speculative/model_drafter.py (1)

315-319:Forwarding generation logits: guard against tensor-device mismatches

Assigning req.py_result.generation_logits directly to target_model_req.py_draft_logits is fine; ensure both tensors reside on the same device (expected CUDA). If PyResult ever stores CPU tensors (e.g., for CPU-only tests), consider moving to the target device or documenting the assumption.

tensorrt_llm/_torch/pyexecutor/sampler.py (1)

860-898:Consistent attribute: use py_seq_slot instead of seq_slot

Elsewhere the code uses py_seq_slot; mixing both can cause subtle bugs if they ever diverge.

Apply:

-            new_tokens[i, request.seq_slot, self.BEAM] = new_token+            new_tokens[i, request.py_seq_slot, self.BEAM] = new_token@@-            new_tokens[num_accepted, request.seq_slot, self.BEAM] = new_token+            new_tokens[num_accepted, request.py_seq_slot, self.BEAM] = new_token

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

tests/unittest/llmapi/test_llm_pytorch.py (1)

234-241:Consider deriving vocab_size_padded instead of hard-coding 32000.

Hard-coding may break with model/tokenizer variants that pad differently.

I can propose a helper to compute padded size from tokenizer vocab and alignment used by the sampler if you confirm the pad multiple.

tensorrt_llm/_torch/modules/rms_norm.py (1)

75-78:Clarify intent: residual holds pre-norm sum (not original residual).

This is correct for fused-add pipelines; consider a brief comment to avoid future regressions.

Apply:

-            if residual is not None:+            if residual is not None:+                # Update residual to the pre-norm sum(hidden_states, residual) for downstream fused-add                 hidden_states = hidden_states + residual.to(torch.float32)                 residual = hidden_states.to(input_dtype)

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: moraxuPR: NVIDIA/TensorRT-LLM#6303File: tests/integration/test_lists/qa/examples_test_list.txt:494-494Timestamp: 2025-07-28T17:06:08.621ZLearning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: pengbowang-nvPR: NVIDIA/TensorRT-LLM#7192File: tests/integration/test_lists/test-db/l0_dgx_b200.yml:56-72Timestamp: 2025-08-26T09:49:04.956ZLearning: In TensorRT-LLM test configuration files, the test scheduling system handles wildcard matching with special rules that prevent duplicate test execution even when the same tests appear in multiple yaml files with overlapping GPU wildcards (e.g., "*b200*" and "*gb200*").

tensorrt_llm/_torch/modules/rms_norm.py (4)

28-30:Make sentinel definition clearer and immutable.

Minor readability/typing tweak: define the alias before the constant and mark the sentinelFinal so type-checkers treat it as immutable.

-from typing import Optional, Tuple, TypeAlias, Union, cast+from typing import Final, Optional, Tuple, TypeAlias, Union, cast@@-    _ARGUMENT_NOT_SPECIFIED_SENTINEL = ...-    _ArgumentNotSpecifiedSentinelType: TypeAlias = EllipsisType+    _ArgumentNotSpecifiedSentinelType: TypeAlias = EllipsisType+    _ARGUMENT_NOT_SPECIFIED_SENTINEL: Final[_ArgumentNotSpecifiedSentinelType] = ...

---

55-63:Streamline the “unspecified residual” logic.

Equivalent but tighter boolean makes intent obvious and avoids the extra flag mutation.

-        return_residual = True-        if residual is self._ARGUMENT_NOT_SPECIFIED_SENTINEL:-            return_residual = False-            residual = None+        return_residual = residual is not self._ARGUMENT_NOT_SPECIFIED_SENTINEL+        if not return_residual:+            residual = None

---

76-79:Avoid an extra allocation in the fallback add.

You can do the add in-place on the FP32 buffer to reduce one temporary tensor.

-            if residual is not None:-                hidden_states = hidden_states + residual.to(torch.float32)-                residual = hidden_states.to(input_dtype)+            if residual is not None:+                hidden_states.add_(residual.to(torch.float32))+                residual = hidden_states.to(input_dtype)

Note: ensure no autograd requirements/in-place aliasing concerns in this inference path.

---

110-116:Fix mutable default and Optional types in group_rms_norm signature.

weights uses a mutable default[], andeps/weight_bias are annotatedOptional but never acceptNone. Clean this to avoid surprises.

-def group_rms_norm(-        inputs: list[torch.Tensor],-        weights: Optional[list[torch.Tensor]] = [],-        eps: Optional[float] = 1e-5,-        weight_bias: Optional[float] = 0.0,+def group_rms_norm(+        inputs: list[torch.Tensor],+        weights: Optional[list[torch.Tensor]] = None,+        eps: float = 1e-5,+        weight_bias: float = 0.0,         kernel: GroupRMSNormKernelSelection = GroupRMSNormKernelSelection.heuristic,         outputs: Optional[list[torch.Tensor]] = None) -> list[torch.Tensor]:@@-    out = outputs+    out = outputs     if out is None:         out = [torch.empty_like(input) for input in inputs]+    if weights is None:+        weights = []

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/modules/rms_norm.py:96-99Timestamp: 2025-08-27T14:41:56.613ZLearning: In tensorrt_llm/_torch/modules/rms_norm.py, the RMSNorm class uses a custom sentinel (_ARGUMENT_NOT_SPECIFIED_SENTINEL) instead of Ellipsis (...) for detecting unspecified optional arguments. Other modules in the codebase may use Ellipsis as a sentinel but do not forward it to RMSNorm methods, so there's no need for backward compatibility with Ellipsis in RMSNorm.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/modules/rms_norm.py:17-17Timestamp: 2025-08-27T14:23:55.545ZLearning: The TensorRT-LLM project requires Python 3.10+ as evidenced by the use of TypeAlias from typing module, match/case statements, and union type | syntax throughout the codebase, despite some documentation still mentioning Python 3.8+.

Learnt from: CRPR: NVIDIA/TensorRT-LLM#0File: CODING_GUIDELINES.md:0-0Timestamp: 2025-08-25T22:42:47.612ZLearning: Applies to **/*.py : Code must target Python 3.8+

tensorrt_llm/_torch/pyexecutor/sampler.py (3)

374-397:Int32 softmax_indices and filtering behavior LGTM

Type hint and greedy softmax-skipping logic align with established TRT-LLM conventions.

---

1-2:Add required NVIDIA copyright header

Per repo guidelines, prepend the NVIDIA header.

Apply:

+# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.+ import dataclasses import enum

---

1018-1064:_apply_embedding_bias(): use id()-based dedup and Long indices for index_select

Tensor keys are unsafe/expensive; index_select requires Long indices.

-        # Indices of unique bias tensors-        bias_to_index: dict[torch.Tensor,-                            int] = defaultdict(provision_bias_index)+        # Indices of unique bias tensors (group by identity, not contents)+        bias_to_index: dict[int, int] = defaultdict(provision_bias_index)+        biases: list[torch.Tensor] = []@@-            if req_bias is not None:+            if req_bias is not None:                 logits_bias_mask[i:(i + steps)] = True-                req_bias_index = bias_to_index[req_bias]+                key = id(req_bias)+                req_bias_index = bias_to_index[key]+                if req_bias_index == len(biases):+                    biases.append(req_bias)                 bias_gather_indices.extend(repeat(req_bias_index, steps))@@-        bias_gather_indices_cuda = torch.tensor(bias_gather_indices,-                                                pin_memory=True,-                                                dtype=torch.int32).to(-                                                    logits.device,-                                                    non_blocking=True)+        bias_gather_indices_cuda = torch.tensor(+            bias_gather_indices, pin_memory=True, dtype=torch.long+        ).to(logits.device, non_blocking=True)@@-        biases_tensor = torch.empty((len(bias_to_index), *req_bias.shape),-                                    pin_memory=True)-        biases_tensor = torch.stack(-            tuple(bias_to_index.keys()),-            out=biases_tensor,-        )+        biases_tensor = torch.stack(+            biases, out=torch.empty((len(biases), *req_bias.shape), pin_memory=True)+        )@@-        biases_tensor_cuda = torch.index_select(biases_tensor_cuda, 0,-                                                bias_gather_indices_cuda)+        biases_tensor_cuda = torch.index_select(+            biases_tensor_cuda, 0, bias_gather_indices_cuda+        )

tensorrt_llm/_torch/pyexecutor/sampler.py (1)

39-56:Confirm minimum supported Python version (uses 3.10+ features)

This file uses kw_only dataclasses, PEP 634 pattern matching (match/case), and PEP 604 unions (X | Y), which require Python 3.10+. Repo guideline says “Python 3.8+”; please confirm runtime targets 3.10+ or adjust.

Would you like a codemod to replace match/case with if/elif and kw_only with runtime-safe alternatives if 3.8/3.9 must be supported?

Also applies to: 119-133, 374-397

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: CRPR: NVIDIA/TensorRT-LLM#0File: CODING_GUIDELINES.md:0-0Timestamp: 2025-08-25T22:42:47.612ZLearning: Applies to **/*.{c,cc,cpp,cxx,h,hh,hpp,hxx,cu,cuh,py} : Prepend the NVIDIA copyright header (current year) to all source files (.cpp, .h, .cu, .py, etc.)

Learnt from: galagamPR: NVIDIA/TensorRT-LLM#6487File: tests/unittest/_torch/auto_deploy/unit/singlegpu/test_ad_trtllm_bench.py:1-12Timestamp: 2025-08-06T13:58:07.506ZLearning: In TensorRT-LLM, test files (files under tests/ directories) do not require NVIDIA copyright headers, unlike production source code files. Test files typically start directly with imports, docstrings, or code.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:368-392Timestamp: 2025-08-27T15:03:57.127ZLearning: In TensorRT-LLM's sampler.py, int32 usage for softmax_indices and related tensor indexing is intentional and should not be changed to int64. The torch.IntTensor type hint is correct for the sample() function's softmax_indices parameter.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:985-1064Timestamp: 2025-08-27T14:52:26.324ZLearning: In TensorRT-LLM's _apply_embedding_bias() method, tensor deduplication should use id() instead of the tensor itself as a dict key to avoid scanning tensor contents and for better performance.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:1229-1238Timestamp: 2025-08-27T14:47:49.339ZLearning: torch.Tensor.dim_order is a public API in PyTorch that returns the dimension order of a tensor, documented at https://docs.pytorch.org/docs/stable/generated/torch.Tensor.dim_order.html. It supports an ambiguity_check parameter that can be a boolean or list of memory formats.

Learnt from: achartierPR: NVIDIA/TensorRT-LLM#6763File: tests/integration/defs/triton_server/conftest.py:16-22Timestamp: 2025-08-11T20:09:24.389ZLearning: In the TensorRT-LLM test infrastructure, the team prefers simple, direct solutions (like hard-coding directory traversal counts) over more complex but robust approaches when dealing with stable directory structures. They accept the maintenance cost of updating tests if the layout changes.

Learnt from: farshadghodsianPR: NVIDIA/TensorRT-LLM#7101File: docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md:36-36Timestamp: 2025-08-21T00:16:56.457ZLearning: TensorRT-LLM container release tags in documentation should only reference published NGC container images. The README badge version may be ahead of the actual published container versions.

Learnt from: moraxuPR: NVIDIA/TensorRT-LLM#6303File: tests/integration/test_lists/qa/examples_test_list.txt:494-494Timestamp: 2025-07-28T17:06:08.621ZLearning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: yibinl-nvidiaPR: NVIDIA/TensorRT-LLM#6506File: examples/models/core/mixtral/requirements.txt:3-3Timestamp: 2025-08-01T15:14:45.673ZLearning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

tensorrt_llm/_torch/pyexecutor/sampler.py (1)

1-2:Add required NVIDIA copyright header (current year).

Repository guideline requires the header on all source files.

Apply:

+ # Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.+ import dataclasses import enum

tensorrt_llm/_torch/pyexecutor/sampler.py (5)

184-191:Guard top-k against k > vocab_size.

Prevents runtime errors when configs overspecify k.

Apply:

-    if top_k > 0:-        values, indices = torch.topk(logits, top_k, dim=-1)+    if top_k > 0:+        top_k = min(top_k, vocab_size)+        values, indices = torch.topk(logits, top_k, dim=-1)

---

249-256:Same guard for top-k in top-k+top-p path.

Apply:

-    if top_k > 0:-        values, _ = torch.topk(logits, top_k, dim=-1)+    if top_k > 0:+        top_k = min(top_k, vocab_size)+        values, _ = torch.topk(logits, top_k, dim=-1)

---

778-783:Make generator device-robust (multi-GPU).

Reinitialize when device changes instead of asserting.

Apply:

-        if self._generator is None:-            # Fallback to a default seed if not set-            self._generator = torch.Generator(device=device)-            self._generator.manual_seed(self._global_seed)-        assert self._generator.device == device+        if self._generator is None or self._generator.device != device:+            self._generator = torch.Generator(device=device)+            self._generator.manual_seed(self._global_seed)         return self._generator

---

1026-1066:Prefer id()-based dedup for clarity; avoid relying on tensor keys.

Identity hashing of tensors is fine, but explicit id() clarifies intent and decouples from tensor-object key semantics.

Apply:

-        bias_to_index: dict[torch.Tensor,-                            int] = defaultdict(provision_bias_index)+        bias_to_index: dict[int, int] = defaultdict(provision_bias_index)+        biases: list[torch.Tensor] = []@@-            if req_bias is not None:+            if req_bias is not None:                 logits_bias_mask[i:(i + steps)] = True-                req_bias_index = bias_to_index[req_bias]+                key = id(req_bias)+                req_bias_index = bias_to_index[key]+                if req_bias_index == len(biases):+                    biases.append(req_bias)                 bias_gather_indices.extend(repeat(req_bias_index, steps))@@-        biases_tensor = torch.empty((len(bias_to_index), *req_bias.shape),-                                    pin_memory=True)-        biases_tensor = torch.stack(-            tuple(bias_to_index.keys()),-            out=biases_tensor,-        )+        biases_tensor = torch.empty((len(biases), *req_bias.shape), pin_memory=True)+        biases_tensor = torch.stack(tuple(biases), out=biases_tensor)

---

52-56:Annotate optional fields as Optional[...] for type correctness.

Current defaults are None but annotations are non-optional.

Apply:

 class SampleState:     scheduled_requests: ScheduledRequests--    device: SampleStateTensors = None-    host: SampleStateTensors = None--    sampler_event: torch.cuda.Event = None+    device: Optional[SampleStateTensors] = None+    host: Optional[SampleStateTensors] = None+    sampler_event: Optional[torch.cuda.Event] = None

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: galagamPR: NVIDIA/TensorRT-LLM#6487File: tests/unittest/_torch/auto_deploy/unit/singlegpu/test_ad_trtllm_bench.py:1-12Timestamp: 2025-08-06T13:58:07.506ZLearning: In TensorRT-LLM, test files (files under tests/ directories) do not require NVIDIA copyright headers, unlike production source code files. Test files typically start directly with imports, docstrings, or code.

Learnt from: CRPR: NVIDIA/TensorRT-LLM#0File: CODING_GUIDELINES.md:0-0Timestamp: 2025-08-25T22:42:47.612ZLearning: Applies to **/*.{c,cc,cpp,cxx,h,hh,hpp,hxx,cu,cuh,py} : Prepend the NVIDIA copyright header (current year) to all source files (.cpp, .h, .cu, .py, etc.)

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:368-392Timestamp: 2025-08-27T15:03:57.127ZLearning: In TensorRT-LLM's sampler.py, int32 usage for softmax_indices and related tensor indexing is intentional and should not be changed to int64. The torch.IntTensor type hint is correct for the sample() function's softmax_indices parameter.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:985-1064Timestamp: 2025-08-27T14:52:26.324ZLearning: In TensorRT-LLM's _apply_embedding_bias() method, tensor deduplication should use id() instead of the tensor itself as a dict key to avoid scanning tensor contents and for better performance.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:1229-1238Timestamp: 2025-08-27T14:47:49.339ZLearning: torch.Tensor.dim_order is a public API in PyTorch that returns the dimension order of a tensor, documented at https://docs.pytorch.org/docs/stable/generated/torch.Tensor.dim_order.html. It supports an ambiguity_check parameter that can be a boolean or list of memory formats.

Learnt from: achartierPR: NVIDIA/TensorRT-LLM#6763File: tests/integration/defs/triton_server/conftest.py:16-22Timestamp: 2025-08-11T20:09:24.389ZLearning: In the TensorRT-LLM test infrastructure, the team prefers simple, direct solutions (like hard-coding directory traversal counts) over more complex but robust approaches when dealing with stable directory structures. They accept the maintenance cost of updating tests if the layout changes.

Learnt from: farshadghodsianPR: NVIDIA/TensorRT-LLM#7101File: docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md:36-36Timestamp: 2025-08-21T00:16:56.457ZLearning: TensorRT-LLM container release tags in documentation should only reference published NGC container images. The README badge version may be ahead of the actual published container versions.

Learnt from: moraxuPR: NVIDIA/TensorRT-LLM#6303File: tests/integration/test_lists/qa/examples_test_list.txt:494-494Timestamp: 2025-07-28T17:06:08.621ZLearning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: ixlmarPR: NVIDIA/TensorRT-LLM#7294File: tensorrt_llm/_torch/pyexecutor/sampler.py:991-1070Timestamp: 2025-08-27T15:37:38.228ZLearning: PyTorch tensors are hashable as dict keys because hash(torch.Tensor) is equivalent to id(torch.Tensor) and uses identity-based hashing, not content-based hashing. This means tensor hashing doesn't scan tensor contents and tensors can be safely used as dictionary keys.