Commit29e2d8f

authored and

committed

[None][feat] Pass KvCacheRetentionConfig to torch LlmRequest (NVIDIA#8634)

Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>

1 parentc471e4a commit29e2d8fCopy full SHA for 29e2d8f

File tree

-1

lines changed

-1

lines changed

Lines changed: 2 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -764,7 +764,8 @@ def executor_request_to_llm_request(`
`764`	`764`	`cache_salt_id=executor_request.cache_salt_id,`
`765`	`765`	`arrival_time=getattr(executor_request,"py_arrival_time",None),`
`766`	`766`	`py_multimodal_data=getattr(executor_request,"py_multimodal_data",`
`767`		`-None))`
	`767`	`+None),`
	`768`	`+kv_cache_retention_config=executor_request.kv_cache_retention_config)`
`768`	`769`	`ifchild_req_ids:`
`769`	`770`	`forchild_idinchild_req_ids:`
`770`	`771`	`llm_request.create_child_request(child_id)`

Comments

(0)