- Notifications
You must be signed in to change notification settings - Fork1k
Closed
Description
Description
Reasoning models such as DeepSeek R1 and the distill variants use a significant amount of tokens while "thinking" before returning a response. It may not be desirable in all cases to pass the reasoning in message history. This feature proposes adding a optional filtering mechanism + out-of-the box filters for users to more easily passing message history. Ex:
frompydantic_ai.filtersimportremove_think_tagsresult=awaitagent.run("Prompt...",# I'm thinking that filters would be something like a Union[str, List[Callable]]...message_history=result.all_messages(filters=remove_think_tags), )
A sample implementation would look something like this:
def_split_think(s:str)->Tuple[str,str]:m=re.match(r"<think>\s*(.*?)\s*</think>\s*(.*)",s,flags=re.DOTALL)ifmandlen(m.groups())==2:return (m.group(1),m.group(2))return ("",s)defremove_think_tags(message:ModelMessage)->ModelMessage:message_copy=deepcopy(message)ifmessage.kind=="response":forpartinmessage_copy.parts:ifisinstance(part,TextPart):_,stripped_content=_split_think(part.content)part.content=stripped_contentreturnmessage_copy
If implemented, thedeepcopy
would be an obvious design discussion on whether filtering should be destructive to the actual message references returned in the agent run result.
If approved, I'd be happy to take a stab at the implementation.
References
No response