- Notifications
You must be signed in to change notification settings - Fork1k
Addbuiltin_tools
toAgent
#1722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
for tool in builtin_tools: | ||
if tool == 'web-search': | ||
self._builtin_tools.append(WebSearchTool()) | ||
else: | ||
self._builtin_tools.append(tool) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It's easier to not have to handle string on the models, so we already do the transformation here.
""" | ||
class UserLocation(TypedDict, total=False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It's easier to handle this in the models if it's aTypedDict
, since it matches the type.
@dataclass | ||
class WebSearchTool(AbstractBuiltinTool): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
From the DX POV, it's nicer for it to be aBaseModel
ordataclass
.
github-actionsbot commentedMay 14, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Docs Preview
|
tools = list(model_settings.get('openai_builtin_tools', [])) + tools | ||
tools = self._get_builtin_tools(model_request_parameters) + tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We should deprecate theopenai_builtin_tools
in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
So we'll add support for FileSearchToolParam and ComputerToolParam as well?
Note thathttps://platform.openai.com/docs/guides/tools-file-search also results in a"type": "file_search_call"
output item.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Do we still want to allow people to use arbitrary built-in tools we haven't created a class for yet by passing the appropriate JSON?
class AbstractBuiltinTool(ABC): | ||
"""A builtin tool that can be used by an agent. | ||
This class is abstract and cannot be instantiated directly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think worth including a sentence here explaining how the code execution works to make use of them — something like "these are passed to the model as part of the ModelRequestParameters" or whatever. (Not sure if that's true, haven't gotten there yet ..). But I imagine it helping someone who is trying to figure out how they are different from normal tools.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM other than tests and docs
Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>
PR Change SummaryEnhanced documentation for the
Modified Files
How can I customize these reviews?Check out theHyperlint AI Reviewer docs for more information on how to customize the review. If you just want to ignore it on this PR, you can add the Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add |
mattbrandman commentedJun 6, 2025
https://github.com/mattbrandman/pydantic-ai/pull/1/files Working on a fork that incorporates the new responses api code interpreter and handles streaming. I'm sure code quality is questionable but feel free to incorporate anything thats useful! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM
tools = list(model_settings.get('openai_builtin_tools', [])) + tools | ||
tools = self._get_builtin_tools(model_request_parameters) + tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
So we'll add support for FileSearchToolParam and ComputerToolParam as well?
Note thathttps://platform.openai.com/docs/guides/tools-file-search also results in a"type": "file_search_call"
output item.
user_location=user_location, | ||
) | ||
) | ||
elif isinstance(tool, CodeExecutionTool): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think we'll want anelse
->raise "Unsupported built-in tool"
here as well (and in OpenAI and Groq, looks like only Google has it currently)
def _get_builtin_tools(self, model_request_parameters: ModelRequestParameters) -> list[responses.ToolParam]: | ||
tools: list[responses.ToolParam] = [] | ||
for tool in model_request_parameters.builtin_tools: | ||
if isinstance(tool, WebSearchTool): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We still need to implement CodeExecutionTool here, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The linked commit I have has a basic implementation. Not sure if it would be best to open a PR into this branch or add it after this is merged. There’s unfortunately a bunch of type errors (or were last I checked) from OpenAI where required fields are empty so need a series of guards that look redundant to checkers
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
No, chat completions doesn't have it.
tools = list(model_settings.get('openai_builtin_tools', [])) + tools | ||
tools = self._get_builtin_tools(model_request_parameters) + tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Do we still want to allow people to use arbitrary built-in tools we haven't created a class for yet by passing the appropriate JSON?
@@ -694,6 +721,20 @@ def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[r | |||
tools += [self._map_tool_definition(r) for r in model_request_parameters.output_tools] | |||
return tools | |||
def _get_builtin_tools(self, model_request_parameters: ModelRequestParameters) -> list[responses.ToolParam]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The case I was thinking of that I mentioned on the call this morning is that someone may useOpenAIModel
withOpenRouterProvider
and an Anthropic model, and it's possible that the built-in tool specifications should then match those for Anthropic rather than those for OpenAI. But from a quick search it doesn't look like OpenRouter currently supports built-in tools (at least there's no doc on it), and when they do they'll probably require a OR-specific specification that they'll translate to the model-specific specs behind the scenes, so we don't have to worry about that for now.
mattbrandman commentedJun 26, 2025
besides having tests pass is there anything else holding back this PR seems like it will grow in importance as more builtin tools are introduced? |
mattbrandman commentedJul 1, 2025
@Kludex not sure if this will be helpful but have a branch based off this with merge conflicts handled and tests running properly. Don't know if I did everything right and also think I should probably have a cleaner one that could be merged into this but my git-foo is failing me late at night will see if I can make a cleaner versionmattbrandman#6 |
@mattbrandman Thanks for creating the#2102 PR!@Kludex is out this week but I'm sure he'll be happy to see the work you've saved him when he's back :) |
mattbrandman commentedJul 1, 2025
@DouweM no problem! I'm happy to have them merged back into here just couldn't figure out a clean PR strategy since I had to merge main to resolve the conflicts. |
I'll continue on#2102. |
No description provided.