When execute_tools_and_side_effects takes long, indeed these hooks would be helpful to understand the exact time spent by LLM. One quick thing I noticed is that probably this implementation doe not yet support streaming patterns.

@rm-openai do you think having these two hooks is good to go?

Thanks for the feedback—here’s a quick run‑through:

1. Why these hooks matter for timing
   By firingon_llm_start immediately beforemodel.get_response(...) andon_llm_end the moment that call returns, we capture a clean window around the pure LLM round‑trip. Anything that happens between those two hooks (tool calls, side‑effects, post‑processing) is clearly outside that window, so long‑running tools no longer obscure where time is spent. You can record timestamps in the hooks (or push spans, metrics, logs, etc.) to drill into LLM latency vs. other work.

2. Streaming support
   @seratch You’re correct that right now we only wrap thebatch (get_response) path. Thestreaming code (run_streamed →model.stream_response) will need a similar hook invocation before the stream starts and once on the finalResponseCompletedEvent. I’ll follow up shortly with a small PR to add those calls.

New: Streaming Support foron_llm_start andon_llm_end Hooks

This update adds full support for theon_llm_start andon_llm_end hooks instreaming mode, ensuring feature parity with non-streaming execution.

1. Implementation (src/agents/run.py)

• Hook calls are now moved inside_run_single_turn_streamed to ensure they receive the correct turn-specific data.
• on_llm_start is triggered just before themodel.stream_response call, with the resolvedsystem_prompt.
• on_llm_end is triggered after theResponseCompletedEvent and once the finalModelResponse is assembled.

2. Unit Tests (tests/test_agent_llm_hooks.py)

• Added a new test:test_streamed_hooks_for_chat_scenario, which verifies that the hooks fire in the correct sequence usingRunner.run_streamed.
• Confirms streaming and non-streaming execution paths are now consistent.

✅ Streaming Example

The hooks now work seamlessly withRunner.run_streamed, enabling real-time insight into LLM usage:

importasynciofromtypingimportAnyfromdotenvimportload_dotenvfromagents.agentimportAgentfromagents.itemsimportItemHelpers,ModelResponsefromagents.lifecycleimportAgentHooks,RunContextWrapperfromagents.runimportRunnerload_dotenv()classLLMTrackerHooks(AgentHooks[Any]):asyncdefon_llm_start(self,context:RunContextWrapper,agent:Agent,system_prompt:str,input_items:Any,    )->None:print(f"[HOOK] on_llm_start: LLM is starting, system_prompt={system_prompt}")asyncdefon_llm_end(self,context:RunContextWrapper,agent:Agent,response:ModelResponse,    )->None:total=response.usage.total_tokensifresponse.usageelseNoneprint(f"[HOOK] on_llm_end: LLM returned, total_tokens={total}")asyncdefmain():agent=Agent(name="MyStreamingAgent",instructions="You are a helpful assistant that can answer questions and perform tasks.",hooks=LLMTrackerHooks(),          )stream=Runner.run_streamed(agent,input="Create a Python script that prints 'Hello, World!'")asyncforeventinstream.stream_events():ifevent.type=="raw_response_event":continueifevent.type=="agent_updated_stream_event":print(f"[EVENT] agent_updated →{event.new_agent.name}")elifevent.type=="run_item_stream_event":item=event.itemifitem.type=="tool_call_item":print("[EVENT] tool_call_item")elifitem.type=="tool_call_output_item":print(f"[EVENT] tool_call_output_item →{item.output}")elifitem.type=="message_output_item":text=ItemHelpers.text_message_output(item)print(f"[EVENT] message_output_item →{text}")print(f"\n[RUN COMPLETE] final_output →{stream.final_output}")if__name__=="__main__":asyncio.run(main())

Thanks again for the valuable feedback,@seratch.
Just a gentle ping for you and@rm-openai to let you know that I've pushed the update to add full streaming support for the LLM hooks, and all CI checks are now passing.
Ready for another look when you have a moment. Thanks!