You signed in with another tab or window.Reload to refresh your session.You signed out in another tab or window.Reload to refresh your session.You switched accounts on another tab or window.Reload to refresh your session.Dismiss alert
[`WrapperToolset`][pydantic_ai.toolsets.WrapperToolset] wraps another toolset and delegates all responsibility to it.
336
336
337
-
To easily chain different modifications, you can also call[`wrap()`][pydantic_ai.toolsets.AbstractToolset.wrap] on any toolset instead of directly constructing an instance of (a subclass of)`WrapperToolset`.
338
-
339
-
`WrapperToolset` is a no-op by default, but enables some useful abilities:
337
+
It is is a no-op by default, but enables some useful abilities:
340
338
341
339
####Changing Tool Execution
342
340
@@ -367,7 +365,7 @@ class LoggingToolset(WrapperToolset):
@@ -438,14 +436,17 @@ If you want to reuse a network connection or session across tool listings and ca
438
436
439
437
###Deferred Toolset
440
438
441
-
A deferred tool is one that will be executed not by Pydantic AI, but by the upstream service that called the agent, such as a web application that supports frontend-defined tools provided to Pydantic AI via a protocol like[AG-UI](https://docs.ag-ui.com/concepts/tools#frontend-defined-tools).
439
+
A deferred tool is one whose result will be produced outside of the Pydantic AI agent run in which it was called, because it depends on an upstream service (or user) or could take longer to generate than it's reasonable to keep the agent process running.
440
+
441
+
Deferred tools enable various use cases:
442
442
443
-
!!! note
444
-
This is not typically something you need to bother with, unless you are implementing support for such a protocol between an upstream tool provider and Pydantic AI.
443
+
- Support client-side tools implemented by a web or app frontend
444
+
- Implement a Human-in-the-Loop flow where the user needs to explicitly provide an "answer" before the run can continue
445
+
- Pass slow tasks off to a background worker or external service that will send a (webhook) notification when the result is ready and the agent run can be continued.
445
446
446
-
When the model calls a deferred tool, the agent run ends with a[`DeferredToolCalls`][pydantic_ai.output.DeferredToolCalls] object containing the deferred tool call names and arguments, whichis expected to be returned to theupstream tool provider. This upstreamserviceis then expected to generate a response for each tool call and starta new Pydantic AI agent run with the message historyand new[`ToolReturnPart`s][pydantic_ai.messages.ToolReturnPart] corresponding to each deferred call, after which the run will continue.
447
+
When the model calls a deferred tool, the agent run ends with a[`DeferredToolCalls`][pydantic_ai.output.DeferredToolCalls] object containing the deferred tool call names and arguments, whichare expected to be returned to the servicethat will (eventually) produce the result(s). Once all the results are ready,a new Pydantic AI agent runcan then be startedwith theoriginal run'smessage historyplus new[`ToolReturnPart`s][pydantic_ai.messages.ToolReturnPart] (or[`RetryPromptPart`s][pydantic_ai.messages.RetryPromptPart] in case of failure) corresponding to each deferred call, after which the run will continue.
447
448
448
-
To enable an agent to call deferred tools, you create a[`DeferredToolset`][pydantic_ai.toolsets.DeferredToolset], pass it a list of[`ToolDefinition`s][pydantic_ai.tools.ToolDefinition], and provide it to the agent using one of the methods described above. Additionally, you need to add`DeferredToolCalls` to the`Agent`'s[output types](output.md#structured-output) so that the agent run's outputtype iscorrectly inferred. Finally, you should handle the possible`DeferredToolCalls`result byreturning it to theupstream tool provider.
449
+
To enable an agent to call deferred tools, you create a[`DeferredToolset`][pydantic_ai.toolsets.DeferredToolset], pass it a list of[`ToolDefinition`s][pydantic_ai.tools.ToolDefinition], and provide it to the agent using one of the methods described above. Additionally, you need to add`DeferredToolCalls` to the`Agent`'s[`output_type`](output.md#structured-output) so that thepossible types of theagent run outputarecorrectly inferred. Finally, you should handle the possible`DeferredToolCalls`output bypassing it to theservice that will produce the results.
449
450
450
451
If your agent can also be used in a context where no deferred tools are available, you will not want to include`DeferredToolCalls` in the`output_type` passed to the`Agent` constructor as you'd have to deal with that type everywhere you use the agent. Instead, you can pass the`toolsets` and`output_type` keyword arguments when you run the agent using[`agent.run()`][pydantic_ai.Agent.run],[`agent.run_sync()`][pydantic_ai.Agent.run_sync],[`agent.run_stream()`][pydantic_ai.Agent.run_stream], or[`agent.iter()`][pydantic_ai.Agent.iter]. Note that while`toolsets` provided at this stage are additional to the toolsets provided to the constructor, the`output_type` overrides the one specified at construction time (for type inference reasons), so you'll need to include the original output types explicitly.
Next, let's definean functionfora hypothetical "run agent" API endpoint that can be called by the frontend and takes a list of messages to send to the model plus adict of frontend toolnames and descriptions. This is where`DeferredToolset` and`DeferredToolCalls` come in:
486
+
Next, let's definea functionthat representsa hypothetical "run agent" API endpoint that can be called by the frontend and takes a list of messages to send to the model plus alist of frontend tooldefinitions. This is where`DeferredToolset` and`DeferredToolCalls` come in:
"""Container for calls of deferred tools. This can be used as an agent's `output_type` and will be used as the output of the agent run if the model called any deferred tools."""
364
+
"""Container for calls of deferred tools. This can be used as an agent's `output_type` and will be used as the output of the agent run if the model called any deferred tools.
365
+
366
+
See [deferred toolset docs](../toolsets.md#deferred-toolset) for more information.
Copy file name to clipboardExpand all lines: pydantic_ai_slim/pydantic_ai/tools.py
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -365,9 +365,9 @@ class ToolDefinition:
365
365
kind:ToolKind=field(default='function')
366
366
"""The kind of tool:
367
367
368
-
- `'function'`: a tool thatcan be executed by Pydantic AI and has its result returned to the model
368
+
- `'function'`: a tool thatwill be executed by Pydantic AI during an agent run and has its result returned to the model
369
369
- `'output'`: a tool that passes through an output value that ends the run
370
-
- `'deferred'`: a toolthatwill beexecuted not byPydantic AI, but by the upstream service that called the agent, such as a web application that supports frontend-defined tools providedtoPydantic AI via e.g. [AG-UI](https://docs.ag-ui.com/concepts/tools#frontend-defined-tools).
370
+
- `'deferred'`: a toolwhose resultwill beproduced outside of thePydantic AI agent run in which it was called, because it depends on an upstream service (or user) or could take longertogenerate than it's reasonable to keep the agent process running.
371
371
When the model calls a deferred tool, the agent run ends with a `DeferredToolCalls` object and a new run is expected to be started at a later point with the message history and new `ToolReturnPart`s corresponding to each deferred call.
"""A toolset that holds deferred toolsthatwill becalled bytheupstream service that called the agent.
17
+
"""A toolset that holds deferred toolswhose resultswill beproduced outside ofthePydantic AI agent run in which they were called.
18
18
19
19
See [toolset docs](../toolsets.md#deferred-toolset), [`ToolDefinition.kind`][pydantic_ai.tools.ToolDefinition.kind], and [`DeferredToolCalls`][pydantic_ai.output.DeferredToolCalls] for more information.