How to add tools to chatbots
Prerequisites
This guide assumes familiarity with the following concepts:
This section will cover how to create conversational agents: chatbots that can interact with other systems and APIs using tools.
note
This how-to guide previously built a chatbot usingRunnableWithMessageHistory. You can access this version of the guide in thev0.2 docs.
As of the v0.3 release of LangChain, we recommend that LangChain users take advantage ofLangGraph persistence to incorporatememory into new LangChain applications.
If your code is already relying onRunnableWithMessageHistory orBaseChatMessageHistory, you donot need to make any changes. We do not plan on deprecating this functionality in the near future as it works for simple chat applications and any code that usesRunnableWithMessageHistory will continue to work as expected.
Please seeHow to migrate to LangGraph Memory for more details.
Setup
For this guide, we'll be using atool calling agent with a single tool for searching the web. The default will be powered byTavily, but you can switch it out for any similar tool. The rest of this section will assume you're using Tavily.
You'll need tosign up for an account on the Tavily website, and install the following packages:
%pip install--upgrade--quiet langchain-openai tavily-python langgraph
import getpass
import os
ifnot os.environ.get("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"]= getpass.getpass("OpenAI API Key:")
ifnot os.environ.get("TAVILY_API_KEY"):
os.environ["TAVILY_API_KEY"]= getpass.getpass("Tavily API Key:")
You will also need your OpenAI key set asOPENAI_API_KEY and your Tavily API key set asTAVILY_API_KEY.
Creating an agent
Our end goal is to create an agent that can respond conversationally to user questions while looking up information as needed.
First, let's initialize Tavily and an OpenAIchat model capable of tool calling:
from langchain_openaiimport ChatOpenAI
from langchain_tavilyimport TavilySearch
tools=[TavilySearch(max_results=10, topic="general")]
# Choose the LLM that will drive the agent
# Only certain models support this
model= ChatOpenAI(model="gpt-4o-mini")
API Reference:ChatOpenAI
To make our agent conversational, we can also specify a prompt. Here's an example:
prompt=(
"You are a helpful assistant. "
"You may not need to use tools for every query - the user may just want to chat!"
)
Great! Now let's assemble our agent using LangGraph's prebuiltcreate_react_agent, which allows you to create atool-calling agent:
from langgraph.prebuiltimport create_react_agent
# prompt allows you to preprocess the inputs to the model inside ReAct agent
# in this case, since we're passing a prompt string, we'll just always add a SystemMessage
# with this prompt string before any other messages sent to the model
agent= create_react_agent(model, tools, prompt=prompt)
API Reference:create_react_agent
Running the agent
Now that we've set up our agent, let's try interacting with it! It can handle both trivial queries that require no lookup:
from langchain_core.messagesimport HumanMessage
agent.invoke({"messages":[HumanMessage(content="I'm Nemo!")]})
API Reference:HumanMessage
{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='40b60204-1af1-40d4-b6a7-b845a2281dd6'),
AIMessage(content='Hi Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 795, 'total_tokens': 806, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsUwqprT2mVdjqu1aaSm1jVVWYVz', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--08282ec6-6d3e-4495-b004-b3b08f3879c3-0', usage_metadata={'input_tokens': 795, 'output_tokens': 11, 'total_tokens': 806, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}
Or, it can use of the passed search tool to get up to date information if needed:
agent.invoke(
{
"messages":[
HumanMessage(
content="What is the current conservation status of the Great Barrier Reef?"
)
],
}
)
{'messages': [HumanMessage(content='What is the current conservation status of the Great Barrier Reef?', additional_kwargs={}, response_metadata={}, id='5240955c-d842-408d-af3d-4ee74db29dbd'),
AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_W37BFkNuZlJu9US1Tl71xpiX', 'function': {'arguments': '{"query":"current conservation status of the Great Barrier Reef","time_range":"year","topic":"general"}', 'name': 'tavily_search'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 32, 'prompt_tokens': 804, 'total_tokens': 836, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsV6EJ7F1vDipoG4dpEiBRZvuTLo', 'service_tier': 'default', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run--5f5b32d7-fb80-4913-a7ec-ca9c5acaa101-0', tool_calls=[{'name': 'tavily_search', 'args': {'query': 'current conservation status of the Great Barrier Reef', 'time_range': 'year', 'topic': 'general'}, 'id': 'call_W37BFkNuZlJu9US1Tl71xpiX', 'type': 'tool_call'}], usage_metadata={'input_tokens': 804, 'output_tokens': 32, 'total_tokens': 836, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}),
ToolMessage(content='{"query": "current conservation status of the Great Barrier Reef", "follow_up_questions": null, "answer": null, "images": [], "results": [{"title": "The Great Barrier Reef: Current Conservation Efforts and Future Outlook", "url": "https://discoverwildscience.com/the-great-barrier-reef-current-conservation-efforts-and-future-outlook-1-279446/", "content": "The Great Barrier Reef, a mesmerizing marvel of nature, stretches over 2,300 kilometers along the northeast coast of Australia. As the largest coral reef system in the world, it is home to an incredible diversity of marine life, including more than 1,500 species of fish and 411 types of hard coral.", "score": 0.6353361, "raw_content": null}, {"title": "Monitoring progress - Protecting the Great Barrier Reef", "url": "https://www.detsi.qld.gov.au/great-barrier-reef/monitoring-progress", "content": "Stay informed about the current state of the Great Barrier Reef through comprehensive monitoring reports and reef report cards. Delve into the scientific research and advancements contributing to reef conservation. Learn about ongoing efforts to track progress and ensure the reef\'s long-term health.", "score": 0.6347929, "raw_content": null}, {"title": "Great Barrier Reef Outlook Report shows that the reef is in serious ...", "url": "https://biodiversitycouncil.org.au/news/great-barrier-reef-outlook-report-shows-that-the-reef-is-in-serious-trouble", "content": "The Great Barrier Reef is in very serious trouble. Climate change is the biggest threat to the reef. Catchment restoration activities that reduce sediment flowing to the reef will aid the health of the reef but cannot match the scale of destruction occurring due to marine heatwaves caused by climate change.", "score": 0.5183761, "raw_content": null}, {"title": "Water pollution threatens Great Barrier Reef\'s survival: new report ...", "url": "https://www.marineconservation.org.au/water-pollution-threatens-great-barrier-reefs-survival-new-report-highlights-funding-need/", "content": "While this investment has supported critical work across the Great Barrier Reef catchments, more funding is needed. At current rates, the target to cut fine sediment by 25% on 2009 levels will not be met until 2047, while the target to reduce dissolved inorganic nitrogen by 60% is not expected to be achieved until 2114.", "score": 0.51383984, "raw_content": null}, {"title": "What is the state of the Great Barrier Reef? - Tangaroa Blue", "url": "https://tangaroablue.org/the-state-of-the-great-barrier-reef/", "content": "The Great Barrier Reef Outlook Report 2024, prepared every five years by the Great Barrier Reef Marine Park Authority, summarises the Reef\'s long-term outlook based on its use, management, and risks.This year\'s report uses data from the Australian Marine Debris Initiative Database to analyse the risks and impacts of marine debris on the Great Barrier Reef and help identify areas for", "score": 0.47489962, "raw_content": null}, {"title": "New report on Great Barrier Reef shows coral cover increases before ...", "url": "https://www.aims.gov.au/information-centre/news-and-stories/new-report-great-barrier-reef-shows-coral-cover-increases-onset-serious-bleaching-cyclones", "content": "Coral cover has increased in all three regions on the Great Barrier Reef and is at regional highs in two of the three regions. But the results come with a note of caution. ... trained scientists during manta tow surveys and is a metric which allows AIMS scientists to provide an overview of the Great Barrier Reef\'s status and keep policy", "score": 0.40330887, "raw_content": null}, {"title": "Cycle of coral bleaching on the Great Barrier Reef now at \'catastrophic ...", "url": "https://www.sydney.edu.au/news-opinion/news/2025/01/21/coral-bleaching-2024-great-barrier-reef-one-tree-island.html", "content": "As the Great Barrier Reef faces increasing threats from climate change, the study calls for a collaborative approach to conservation that involves local communities, scientists and policymakers. Dr Shawna Foo , a Sydney Horizon Fellow and co-author of the study, said: \\"Seeing the impacts on a reef that has largely avoided mass bleaching until", "score": 0.3759361, "raw_content": null}, {"title": "Great Barrier Reef Outlook Report 2024: An ecosystem under pressure", "url": "https://icriforum.org/gbr-outlook-report-2024/", "content": "The 2024 Great Barrier Reef Outlook Report is the fourth in a series of comprehensive five-yearly reports on the Reef\'s health, pressures, management, and potential future. It found climate-driven threats such as warming oceans and severe cyclones have been compounding other impacts from crown-of-thorns starfish outbreaks, poor water quality", "score": 0.34634283, "raw_content": null}, {"title": "UNESCO expresses \'utmost concern\' at the state of the Great Barrier Reef", "url": "https://theconversation.com/unesco-expresses-utmost-concern-at-the-state-of-the-great-barrier-reef-257638", "content": "This 2017 photo from Ribbon Reef, near Cairns, shows what a healthy reef looks like. J Summerling/AP Poor water quality persists. Poor water quality is a major issue on the Great Barrier Reef.", "score": 0.31069487, "raw_content": null}, {"title": "Reef health updates | Reef Authority - gbrmpa", "url": "https://www2.gbrmpa.gov.au/learn/reef-health/reef-health-updates", "content": "As the lead managers of the Great Barrier Reef, the Reef Authority keeps an eye on the Reef year-round — with efforts stepped up over summer, a typically high-risk period from extreme weather. The Reef Authority releases updates on the health of Reef which includes; sea surface temperatures, rainfall and floods, cyclones, crown-of-thorns", "score": 0.18051112, "raw_content": null}], "response_time": 2.07}', name='tavily_search', id='cbf7ae84-1df7-4ead-b00d-f8fba2152720', tool_call_id='call_W37BFkNuZlJu9US1Tl71xpiX'),
AIMessage(content='The current conservation status of the Great Barrier Reef is concerning. The reef is facing significant threats primarily due to climate change, which is causing marine heatwaves and coral bleaching. A report highlights that while there have been some local efforts in conservation, such as catchment restoration to reduce sediment flow, these cannot keep pace with the destruction caused by climate impacts. Recent findings from the 2024 Great Barrier Reef Outlook Report indicate that climate-driven phenomena like warming oceans and severe cyclones are exacerbating other pressures, such as crown-of-thorns starfish outbreaks and poor water quality.\n\nSome reports have indicated that coral cover has increased in certain regions of the reef, but overall, the health of the reef remains in serious decline. There’s an urgent call for more funding and collaborative efforts between local communities, scientists, and policymakers to enhance conservation measures.\n\nFor more detailed information, you can refer to these articles:\n- [The Great Barrier Reef: Current Conservation Efforts and Future Outlook](https://discoverwildscience.com/the-great-barrier-reef-current-conservation-efforts-and-future-outlook-1-279446/)\n- [Great Barrier Reef Outlook Report 2024: An ecosystem under pressure](https://icriforum.org/gbr-outlook-report-2024/)', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 255, 'prompt_tokens': 2208, 'total_tokens': 2463, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVAxeGL7PKGVkb2DieFPE0ZPgor', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--71441b27-81a0-427f-8784-b2ea674bebd4-0', usage_metadata={'input_tokens': 2208, 'output_tokens': 255, 'total_tokens': 2463, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}
Conversational responses
Because our prompt contains a placeholder for chat history messages, our agent can also take previous interactions into account and respond conversationally like a standard chatbot:
from langchain_core.messagesimport AIMessage, HumanMessage
agent.invoke(
{
"messages":[
HumanMessage(content="I'm Nemo!"),
AIMessage(content="Hello Nemo! How can I assist you today?"),
HumanMessage(content="What is my name?"),
],
}
)
API Reference:AIMessage |HumanMessage
{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='8a67dea0-acd8-40f9-8c28-292c5f81c05f'),
AIMessage(content='Hello Nemo! How can I assist you today?', additional_kwargs={}, response_metadata={}, id='92a2533e-5c62-4cbe-80f1-302f5f1caf28'),
HumanMessage(content='What is my name?', additional_kwargs={}, response_metadata={}, id='efa8c3d3-86d7-428f-985e-a3aadd6504bc'),
AIMessage(content='Your name is Nemo!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 818, 'total_tokens': 824, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVIf5MX5jXUEjYCorT5bWYzc7iu', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a1a32c7d-8066-4954-86f9-3a8f43fcb48d-0', usage_metadata={'input_tokens': 818, 'output_tokens': 6, 'total_tokens': 824, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}
If preferred, you can also add memory to the LangGraph agent to manage the history of messages. Let's redeclare it this way:
from langgraph.checkpoint.memoryimport MemorySaver
memory= MemorySaver()
agent= create_react_agent(model, tools, prompt=prompt, checkpointer=memory)
API Reference:MemorySaver
agent.invoke(
{"messages":[HumanMessage("I'm Nemo!")]},
config={"configurable":{"thread_id":"1"}},
)
{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='31c2249a-13eb-4040-b56d-0c8746fa158e'),
AIMessage(content='Hello, Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 12, 'prompt_tokens': 795, 'total_tokens': 807, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVRB0FItvtPawTTIAjNwgmlQFFw', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a9703ca1-de4c-4f76-b622-9683d86ca777-0', usage_metadata={'input_tokens': 795, 'output_tokens': 12, 'total_tokens': 807, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}
And then if we rerun our wrapped agent executor:
agent.invoke(
{"messages":[HumanMessage("What is my name?")]},
config={"configurable":{"thread_id":"1"}},
)
{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='31c2249a-13eb-4040-b56d-0c8746fa158e'),
AIMessage(content='Hello, Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 12, 'prompt_tokens': 795, 'total_tokens': 807, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVRB0FItvtPawTTIAjNwgmlQFFw', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a9703ca1-de4c-4f76-b622-9683d86ca777-0', usage_metadata={'input_tokens': 795, 'output_tokens': 12, 'total_tokens': 807, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}),
HumanMessage(content='What is my name?', additional_kwargs={}, response_metadata={}, id='0cde6457-8d4d-45d5-b175-ad846018c4d2'),
AIMessage(content='Your name is Nemo! How can I help you today, Nemo?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 819, 'total_tokens': 834, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVTa1plxGPNitbOcw7YVTFdmz1e', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--1d742bc1-5839-4837-b6f4-9a6b92fa6897-0', usage_metadata={'input_tokens': 819, 'output_tokens': 15, 'total_tokens': 834, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}
ThisLangSmith trace shows what's going on under the hood.
Further reading
For more on how to build agents, check theseLangGraph guides:
For more on tool usage, you can also check outthis use case section.