Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Add parallel tool execution to ReAct#8999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
Copilot wants to merge5 commits intomain
base:main
Choose a base branch
Loading
fromcopilot/add-parallel-tool-execution

Conversation

Copy link
Contributor

CopilotAI commentedOct 30, 2025•
edited
Loading

Parallel Tool Execution for ReAct

This PR adds support for parallel tool execution in the ReAct module, allowing the LLM to request multiple tool calls in a single turn and execute them concurrently.

Plan

  • Explore the codebase and understand current ReAct implementation
  • Understand existing Tool and ToolCalls classes
  • Run existing tests to ensure baseline functionality
  • Modify ReAct signature to support multiple tool calls output
  • Implement parallel tool execution in syncforward method using ThreadPoolExecutor
  • Implement parallel tool execution in asyncaforward method using asyncio.gather
  • Add comprehensive tests for parallel tool execution (both sync and async)
  • Test with multiple tools being called simultaneously
  • Ensure backward compatibility with single tool calls
  • Update existing tests to new format
  • Run linting and all tests
  • Request code review
  • Address code review feedback
  • Merge all tests into single test file
  • Use ToolCalls class for explicit schema definition

Summary

This implementation allows the LLM to request multiple tool calls in a single turn, which are then executed concurrently:

  • Sync execution: UsesThreadPoolExecutor for parallel execution
  • Async execution: Usesasyncio.gather for parallel execution
  • Trajectory format: Updated to support multiple tool calls and observations per iteration
  • Backward compatible: Works with single tool calls as well
  • Error handling: Each tool execution is isolated, errors in one tool don't affect others
  • Consolidated tests: All tests now intest_react.py for better organization
  • Explicit schema: UsesToolCalls class for type annotation

Key Changes

  1. Signature Update: Changed fromnext_tool_name +next_tool_args tonext_tool_calls (ToolCalls type)
  2. Parallel Execution: Implemented using ThreadPoolExecutor (sync) and asyncio.gather (async)
  3. Trajectory Format: Storestool_calls_X (list) andobservations_X (list of dicts) instead of individual fields
  4. Comprehensive Tests: Added 7 new tests covering various parallel execution scenarios, all intest_react.py
  5. Updated Tests: All 6 existing tests updated to use new format
  6. Explicit Schema: Uses ToolCalls class for better type safety and schema definition
Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature] Parallel tool execution</issue_title>
<issue_description>### What feature would you like to see?

Is there any plan to make parallel tool execution feature for React etc?

Would you like to contribute?

  • Yes, I'd like to help implement this.
  • No, I just want to request it.

Additional Context

No response</issue_description>

<agent_instructions>Can you support parallel tool calling in ReAct? Notice that we need to support both sync and async apis</agent_instructions>

Comments on the Issue (you are@copilot in this section)

@TomeHirataHi,@quancore? Did you face a situation where LLM requests multiple tool calls in one turn, and each tool call took time? Would love to know more details to decide the priority!

💬 We'd love your input! Share your thoughts on Copilot coding agent in our2 minute survey.

CopilotAI self-assigned thisOct 30, 2025
CopilotAIand others added2 commitsOctober 30, 2025 08:44
- Modified ReAct signature to accept next_tool_calls list instead of single tool_name/tool_args- Implemented parallel execution for sync using ThreadPoolExecutor- Implemented parallel execution for async using asyncio.gather- Updated trajectory format to support multiple tool calls and observations per iteration- Added comprehensive tests for parallel tool execution (both sync and async)- Updated all existing tests to use new format- All tests passing with proper lintingCo-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
- Remove unnecessary list() conversion from asyncio.gather return- Remove unused elapsed_time calculations from tests- All tests still passing with proper lintingCo-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
CopilotAI changed the title[WIP] Add parallel tool execution feature for ReactAdd parallel tool execution to ReActOct 30, 2025
CopilotAI requested a review fromTomeHirataOctober 30, 2025 08:51
Copilot finished work on behalf ofTomeHirataOctober 30, 2025 08:51
- Consolidated all parallel tool execution tests into test_react.py- Removed test_react_parallel.py as requested in code review- Added asyncio and time imports to support parallel tests- All 14 tests passing with proper lintingCo-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
CopilotAI requested a review fromTomeHirataOctober 30, 2025 10:10
Copilot finished work on behalf ofTomeHirataOctober 30, 2025 10:10
@chenmoneygithub
Copy link
Collaborator

I am worrying that the change to prompt can lead to performance regression, since it might be possible LM does a worse job selecting multiple tools in a bulk. We can run benchmarks to verify it, or probably we should introduce a new mode for parallel execution?

@copilot what's your thought on this?

TomeHirata reacted with thumbs up emoji

Copy link
ContributorAuthor

I am worrying that the change to prompt can lead to performance regression, since it might be possible LM does a worse job selecting multiple tools in a bulk. We can run benchmarks to verify it, or pr...

You raise a valid concern. The current implementation does change the prompt for all ReAct instances, which could impact performance in single-tool scenarios.

A few thoughts on addressing this:

  1. The change is actually backward compatible in practice: The LLM can still return a single tool call by providing a list with one element[{"name": "tool", "args": {...}}]. The prompt explicitly states "You can call multiple tools in parallel" (line 59) rather than requiring it, so the LLM can choose to call just one tool.

  2. Potential solutions if benchmarks show regression:

    • Add an optionalparallel_tools=True/False parameter to__init__ that switches between the old and new prompt/signature
    • Make parallel the default but allow users to opt-out for simpler cases
    • Keep current implementation but add better guidance in the prompt to help LLM decide when to use single vs multiple tools

  3. Recommendation: I'd suggest running benchmarks on common single-tool scenarios first. If there's measurable regression, we can add a mode parameter. If performance is similar, the simpler unified approach may be preferable.

Would you like me to implement an optional mode parameter, or would you prefer to evaluate performance first?

Copilot finished work on behalf ofchenmoneygithubOctober 30, 2025 19:04
.append("next_thought",dspy.OutputField(),type_=str)
.append("next_tool_name",dspy.OutputField(),type_=Literal[tuple(tools.keys())])
.append("next_tool_args",dspy.OutputField(),type_=dict[str,Any])
.append("next_tool_calls",dspy.OutputField(),type_=list[dict[str,Any]])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@copilot can we useToolCalls class to define the schema explicitly?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Done! Changed the type annotation fromlist[dict[str, Any]] toToolCalls and updated the_parse_tool_calls method to handle ToolCalls objects. All tests passing. (commite0833e3)

@TomeHirata
Copy link
Collaborator

TomeHirata commentedNov 6, 2025•
edited
Loading

@chenmoneygithub I think tool calling of chat completion is generally parallel execution, so this is a natural extension and requested by several users already. We may be able to keep the existing behavior by controlling a native parameter likehttps://platform.openai.com/docs/guides/function-calling#parallel-function-calling?

- Changed type annotation from list[dict[str, Any]] to ToolCalls- Updated _parse_tool_calls to handle ToolCalls objects- Added ToolCalls to imports- All tests passing with proper lintingCo-authored-by: TomeHirata <33407409+TomeHirata@users.noreply.github.com>
CopilotAI requested a review fromTomeHirataNovember 6, 2025 04:48
Copilot finished work on behalf ofTomeHirataNovember 6, 2025 04:48
Ju-usc added a commit to Ju-usc/dspy that referenced this pull requestNov 14, 2025
Document that this is a workaround for ReAct's multiple predictorcalls with partial trajectories. After PRstanfordnlp#8999 merges, we shouldtest if we can remove this and use extract predictor trace directly.
returnf"Execution error in{tool_name}:{_fmt_exc(err)}"

# Execute tools in parallel using asyncio.gather
observations=awaitasyncio.gather(*[execute_single_tool(tc)fortcintool_calls])
Copy link

@gauravkumar37gauravkumar37Nov 16, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@copilot Use the newerToolCall.execute()
cc:@chenmoneygithub@TomeHirata

@gauravkumar37
Copy link

How do you feel about this PR?@TomeHirata@chenmoneygithub
I testing this and it works well, interdependent tools are called sequentially and independent tool calls are executed parallely.
Of course, I can't comment on whether it will lead to performance regressions with single tool calls.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@chenmoneygithubchenmoneygithubAwaiting requested review from chenmoneygithub

@TomeHirataTomeHirataAwaiting requested review from TomeHirata

1 more reviewer

@gauravkumar37gauravkumar37gauravkumar37 left review comments

Reviewers whose approvals may not affect merge requirements

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature] Parallel tool execution

4 participants

@chenmoneygithub@TomeHirata@gauravkumar37
[8]ページ先頭
©2009-2025 Movatter.jp