Introduction

Agent Framework / shim to use Pydantic with LLMs

PydanticAI is a Python agent framework designed to make it less painful to build production grade applications with Generative AI.

FastAPI revolutionized web development by offering an innovative and ergonomic design, built on the foundation ofPydantic.

Similarly, virtually every agent framework and LLM library in Python uses Pydantic, yet when we began to use LLMs inPydantic Logfire, we couldn't find anything that gave us the same feeling.

We built PydanticAI with one simple aim: to bring that FastAPI feeling to GenAI app development.

Why use PydanticAI

• Built by the Pydantic Team:Built by the team behindPydantic (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor and many more).

• Model-agnostic:Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral, and there is a simple interface to implement support forother models.

• Pydantic Logfire Integration:Seamlesslyintegrates withPydantic Logfire for real-time debugging, performance monitoring, and behavior tracking of your LLM-powered applications.

• Type-safe:Designed to maketype checking as powerful and informative as possible for you.

• Python-centric Design:Leverages Python's familiar control flow and agent composition to build your AI-driven projects, making it easy to apply standard Python best practices you'd use in any other (non-AI) project.

• Structured Responses:Harnesses the power ofPydantic tovalidate and structure model outputs, ensuring responses are consistent across runs.

• Dependency Injection System:Offers an optionaldependency injection system to provide data and services to your agent'ssystem prompts,tools andoutput validators.This is useful for testing and eval-driven iterative development.

• Streamed Responses:Provides the ability tostream LLM responses continuously, with immediate validation, ensuring real time access to validated outputs.

• Graph Support:Pydantic Graph provides a powerful way to define graphs using typing hints, this is useful in complex applications where standard control flow can degrade to spaghetti code.

Hello World Example

Here's a minimal example of PydanticAI:

frompydantic_aiimportAgentagent=Agent(# (1)!'google-gla:gemini-1.5-flash',system_prompt='Be concise, reply with one sentence.',# (2)!)result=agent.run_sync('Where does "hello world" come from?')# (3)!print(result.output)"""The first known use of "hello, world" was in a 1974 textbook about the C programming language."""

1. We configure the agent to useGemini 1.5's Flash model, but you can also set the model when running the agent.
2. Register a staticsystem prompt using a keyword argument to the agent.
3. Run the agent synchronously, conducting a conversation with the LLM.

(This example is complete, it can be run "as is")

The exchange should be very short: PydanticAI will send the system prompt and the user query to the LLM, the model will return a text response.

Not very interesting yet, but we can easily add "tools", dynamic system prompts, and structured responses to build more powerful agents.

Tools & Dependency Injection Example

Here is a concise example using PydanticAI to build a support agent for a bank:

fromdataclassesimportdataclassfrompydanticimportBaseModel,Fieldfrompydantic_aiimportAgent,RunContextfrombank_databaseimportDatabaseConn@dataclassclassSupportDependencies:# (3)!customer_id:intdb:DatabaseConn# (12)!classSupportOutput(BaseModel):# (13)!support_advice:str=Field(description='Advice returned to the customer')block_card:bool=Field(description="Whether to block the customer's card")risk:int=Field(description='Risk level of query',ge=0,le=10)support_agent=Agent(# (1)!'openai:gpt-4o',# (2)!deps_type=SupportDependencies,output_type=SupportOutput,# (9)!system_prompt=(# (4)!'You are a support agent in our bank, give the ''customer support and judge the risk level of their query.'),)@support_agent.system_prompt# (5)!asyncdefadd_customer_name(ctx:RunContext[SupportDependencies])->str:customer_name=awaitctx.deps.db.customer_name(id=ctx.deps.customer_id)returnf"The customer's name is{customer_name!r}"@support_agent.tool# (6)!asyncdefcustomer_balance(ctx:RunContext[SupportDependencies],include_pending:bool)->float:"""Returns the customer's current account balance."""# (7)!returnawaitctx.deps.db.customer_balance(id=ctx.deps.customer_id,include_pending=include_pending,)...# (11)!asyncdefmain():deps=SupportDependencies(customer_id=123,db=DatabaseConn())result=awaitsupport_agent.run('What is my balance?',deps=deps)# (8)!print(result.output)# (10)!"""    support_advice='Hello John, your current account balance, including pending transactions, is $123.45.' block_card=False risk=1    """result=awaitsupport_agent.run('I just lost my card!',deps=deps)print(result.output)"""    support_advice="I'm sorry to hear that, John. We are temporarily blocking your card to prevent unauthorized transactions." block_card=True risk=8    """

1. Thisagent will act as first-tier support in a bank. Agents are generic in the type of dependencies they accept and the type of output they return. In this case, the support agent has typeAgent[SupportDependencies,SupportOutput].
2. Here we configure the agent to useOpenAI's GPT-4o model, you can also set the model when running the agent.
3. TheSupportDependencies dataclass is used to pass data, connections, and logic into the model that will be needed when runningsystem prompt andtool functions. PydanticAI's system of dependency injection provides atype-safe way to customise the behavior of your agents, and can be especially useful when runningunit tests and evals.
4. Staticsystem prompts can be registered with thesystem_prompt keyword argument to the agent.
5. Dynamicsystem prompts can be registered with the@agent.system_prompt decorator, and can make use of dependency injection. Dependencies are carried via theRunContext argument, which is parameterized with thedeps_type from above. If the type annotation here is wrong, static type checkers will catch it.
6. tool let you register functions which the LLM may call while responding to a user. Again, dependencies are carried viaRunContext, any other arguments become the tool schema passed to the LLM. Pydantic is used to validate these arguments, and errors are passed back to the LLM so it can retry.
7. The docstring of a tool is also passed to the LLM as the description of the tool. Parameter descriptions areextracted from the docstring and added to the parameter schema sent to the LLM.
8. Run the agent asynchronously, conducting a conversation with the LLM until a final response is reached. Even in this fairly simple case, the agent will exchange multiple messages with the LLM as tools are called to retrieve an output.
9. The response from the agent will, be guaranteed to be aSupportOutput, if validation failsreflection will mean the agent is prompted to try again.
10. The output will be validated with Pydantic to guarantee it is aSupportOutput, since the agent is generic, it'll also be typed as aSupportOutput to aid with static type checking.
11. In a real use case, you'd add more tools and a longer system prompt to the agent to extend the context it's equipped with and support it can provide.
12. This is a simple sketch of a database connection, used to keep the example short and readable. In reality, you'd be connecting to an external database (e.g. PostgreSQL) to get information about customers.
13. ThisPydantic model is used to constrain the structured data returned by the agent. From this simple definition, Pydantic builds the JSON Schema that tells the LLM how to return the data, and performs validation to guarantee the data is correct at the end of the run.

Instrumentation with Pydantic Logfire

To understand the flow of the above runs, we can watch the agent in action using Pydantic Logfire.

To do this, we need to set up logfire, and add the following to our code:

...frompydantic_aiimportAgent,RunContextfrombank_databaseimportDatabaseConnimportlogfirelogfire.configure()# (1)!logfire.instrument_asyncpg()# (2)!...support_agent=Agent('openai:gpt-4o',deps_type=SupportDependencies,output_type=SupportOutput,system_prompt=('You are a support agent in our bank, give the ''customer support and judge the risk level of their query.'),instrument=True,)

1. Configure logfire, this will fail if project is not set up.
2. In our demo,DatabaseConn usesasyncpg to connect to a PostgreSQL database, sologfire.instrument_asyncpg() is used to log the database queries.

That's enough to get the following view of your agent in action:

SeeMonitoring and Performance to learn more.

llms.txt

The PydanticAI documentation is available in thellms.txt format.This format is defined in Markdown and suited for large language models.

Two formats are available:

• llms.txt: a file containing a brief description of the project, along with links to the different sections of the documentation. The structure of this file is described in detailshere.
• llms-full.txt: Similar to thellms.txt file, but every link content is included. Note that this file may be too large for some LLMs.

As of today, these filescannot be natively leveraged by LLM frameworks or IDEs. Alternatively,anMCP server can be implemented to properly parse thellms.txtfile.

Next Steps

To try PydanticAI yourself, follow the instructionsin the examples.

Read thedocs to learn more about building applications with PydanticAI.

Read theAPI Reference to understand PydanticAI's interface.