validate OpenAI responses#2226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open

samuelcolvin wants to merge2 commits intomain

base:main

Choose a base branch

Open

+212 −2

Conversation

Copy link

Member

@DouweM @Kludex I think there were some issues about this, can you remember?

maybefixes#527?

Although the OpenAI SDK claims to return a Pydantic model (ChatCompletion) from the chat completions endpoint:

it hasn't actually performed validation (presumably they're creating the model withmodel_construct or something?!)
if the endpoint returns plain text, the return type is a string

Thus we validate it fully here.

validate OpenAI responses

2a760e0

samuelcolvin mentioned this pull request

Open

Copy link

commit:	`e1bc138`
Preview URL:	https://904ab5a1-pydantic-ai-previews.pydantic.workers.dev

improve comment

e1bc138

Kludex reviewed

Copy link

Member

The reason will be displayed to describe this comment to others.Learn more.

How are you generating those tests? Manually? 🤔


		async def test_valid_response(env: TestEnv, allow_model_requests: None):
		"""VCR recording is of a valid response."""
		env.set('OPENAI_API_KEY', 'foobar')