pls pls see to it urgently that linting errors are solved.

pls pls see to it urgently that linting errors are solved.

yea i dont think those failures are on what i added, i think therye elsewhere :)

can we get this landed?

This is a nice to have, and unblocks core functionality. Which runway will this land on?

since it's supported by the others will add to the other clients as well

looks like mostof them already supprt / have it integrated, im not sure how it should look for gemini/cohere, cc@sydney-runkle

+1 request for review, are additional changes needed to get this functionality merged? Thank you!

confused on what more is needed here ?

Thanks@sydney-runkle , will try to get these changes in tommorow; I don't have a cohere , groq, or anthropic sub so unsure how to test those ; I only have access to llama based models and openAI so I might need assist on those; anthropic looks pretty straightforward I can prob get that one in also, and groq is llama so can prob swing that one as well, never touched cohere before though

@webcoderz, you can look at their API/sdk docs to figure out what values are accepted + supported.

Given the demand for this, I'm also happy to pick this up tomorrow and get this into our next release.

Would be super-useful to my use case, too, to get this merged and released, anything I can help with?

As of when I checked this AM vLLM does support reasoning models + tool calls now but sadly tool_choice=required support hasn't been included yet.

IMO I do agree with folks that more flexibility over tool_choice mapping is desirable given current SoTA.@Kludex , what do you think? Can this be merged?

tool_choice: required in vLLM just got merged. Looking forward for a fix! Thank you all!

Would be super-useful to my use case, too, to get this merged and released, anything I can help with?

As of when I checked this AM vLLM does support reasoning models + tool calls now but sadly tool_choice=required support hasn't been included yet.
IMO I do agree with folks that more flexibility over tool_choice mapping is desirable given current SoTA.@Kludex , what do you think? Can this be merged?

tool_choice: required in vLLM just got merged. Looking forward for a fix! Thank you all!

Was merged to main in vLLM, no actual release yet - but i'll drop a post here once it finished compiling from source if it now works

Would be super-useful to my use case, too, to get this merged and released, anything I can help with?

As of when I checked this AM vLLM does support reasoning models + tool calls now but sadly tool_choice=required support hasn't been included yet.
IMO I do agree with folks that more flexibility over tool_choice mapping is desirable given current SoTA.@Kludex , what do you think? Can this be merged?

tool_choice: required in vLLM just got merged. Looking forward for a fix! Thank you all!

Was merged to main in vLLM, no actual release yet - but i'll drop a post here once it finished compiling from source if it now works

It doesn't work with reasoning models unfortunately, I add a fix to15627.

Neither XGrammar nor Outlines seem to work out of the box with json subschema on some models, but at least thetool_choice="required" is working as intended with vLLM main and llama3 architectures in my tests

Neither XGrammar nor Outlines seem to work out of the box with json subschema on some models, but at least thetool_choice="required" is working as intended with vLLM main and llama3 architectures in my tests

Do you also got same issue?

json.decoder.JSONDecodeError: Expecting ':' delimiter: line 2 column 39 (char 39)

I think we should still get this pushed through if possible to standardize how to handle tool_choice , unless there’s some alternative methods@Kludex

p.s. thanks for your patience with this :)

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?

I still need to add tests, but I'd like to add vllm to our test suite.

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?

I still need to add tests, but I'd like to add vllm to our test suite.

Your changes work, issues were with tool call json subschema parsing in outlines and xgrammar with vLLM if one adds mcp servers andresult_type at the same time to an Agent. If you decide to add vLLM to the test suite, i am happy to contribute my test cases, since we run these in prod anyway. See myexample above

Edit: to clarify, the tool_choice="required" works without this PR now - the examples above are still broken in tool parser libs but are unrelated to pydantic-ai or this PR

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?

I still need to add tests, but I'd like to add vllm to our test suite.

that’s a great idea

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?
I still need to add tests, but I'd like to add vllm to our test suite.

that’s a great idea

So nothing missing? 😅

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?
I still need to add tests, but I'd like to add vllm to our test suite.

that’s a great idea

So nothing missing? 😅

Haha sorry,was on a meeting and trying to multitask .. no it ended up working once vLLM merged their PR for tool_choice=“required” a couple weeks ago :) there’s so many comments here and that got lost, theres still a slight problem on their side with parsing nested pydantic models for structured output but I think that’s WIP by@theobjectivedad but if you have some suggestions on how to handle that would be helpful

@webcoderz is there something missing here? I read somewhere above that my changes were not correct, is that the case?
I still need to add tests, but I'd like to add vllm to our test suite.

Your changes work, issues were with tool call json subschema parsing in outlines and xgrammar with vLLM if one adds mcp servers andresult_type at the same time to an Agent. If you decide to add vLLM to the test suite, i am happy to contribute my test cases, since we run these in prod anyway. See myexample above

Edit: to clarify, the tool_choice="required" works without this PR now - the examples above are still broken in tool parser libs but are unrelated to pydantic-ai or this PR

just checked this and yea seems fine here, the main issue is in xgrammar vllm side now

@webcoderz Can you please rebase this on main?

@webcoderz Can you please rebase this on main?

Sure will work on this today/tommorow, it’s been a while :)

bump, this also blocking usage of AWS Bedrock llama4 models

so far I've appliedthis as a workaround, but not clear if it covers all use cases.

bump, this also blocking usage of AWS Bedrock llama4 models

so far I've appliedthis as a workaround, but not clear if it covers all use cases.

Thanks for bumping this@okigan i got tied up doing other stuff , but will recircle back to this

@webcoderz Great, please let me know when it's ready for review again!

This PR is stale, and will be closed in 3 days if no reply is received.