Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Harness LLMs with Multi-Agent Programming

License

NotificationsYou must be signed in to change notification settings

langroid/langroid

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2,386 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Logo

PyPI - VersionDownloadsPytestcodecovMulti-Architecture DockerHub

Static BadgeOpen in ColabDiscordSubstack

GitMCP

Langroid is an intuitive, lightweight, extensible and principledPython framework to easily build LLM-powered applications, from CMU and UW-Madison researchers.You set up Agents, equip them with optional components (LLM,vector-store and tools/functions), assign them tasks, and have themcollaboratively solve a problem by exchanging messages.This Multi-Agent paradigm is inspired by theActor Framework(but you do not need to know anything about this!).

Langroid is a fresh take on LLM app-development, where considerable thought has goneinto simplifying the developer experience;it does not useLangchain, or any other LLM framework,and works withpractically any LLM.

🔥 ✨ A Claude Codeplugin is available toaccelerate Langroid development with built-in patterns and best practices.

🔥 Read the (WIP)overview of the langroid architecture,and aquick tour of Langroid.

🔥 MCP Support: Allow any LLM-Agent to leverage MCP Servers via Langroid's simpleMCP tool adapter that convertsthe server's tools into Langroid'sToolMessage instances.

📢 Companies are using/adapting Langroid inproduction. Here is a quote:

Nullify uses AI Agents for secure software development.It finds, prioritizes and fixes vulnerabilities. We have internally adapted Langroid's multi-agent orchestration framework in production, after evaluating CrewAI, Autogen, LangChain, Langflow, etc. We found Langroid to be far superior to those frameworks in terms of ease of setup and flexibility. Langroid's Agent and Task abstractions are intuitive, well thought out, and provide a great developer experience. We wanted the quickest way to get something in production. With other frameworks it would have taken us weeks, but with Langroid we got to good results in minutes. Highly recommended!
-- Jacky Wong, Head of AI at Nullify.

🔥 See thisIntro to Langroidblog post from the LanceDB team

🔥 Just published in ML for Healthcare (2024): a Langroid-based Multi-Agent RAG system forpharmacovigilance, seeblog post

We welcome contributions: See thecontributions documentfor ideas on what to contribute.

Are you building LLM Applications, or want help with Langroid for your company,or want to prioritize Langroid features for your company use-cases?Prasad Chalasani is available for consulting(advisory/development): pchalasani at gmail dot com.

Sponsorship is also accepted viaGitHub Sponsors

Questions, Feedback, Ideas? Join us onDiscord!

Quick glimpse of coding with Langroid

This is just a teaser; there's much more, like function-calling/tools,Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent(RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more.See the Langroid Quick-StartColabthat builds up to a 2-agent information-extraction example using the OpenAI ChatCompletion API.See also thisversion that uses the OpenAI Assistants API instead.

🔥 just released!Examplescript showing how you can use Langroid multi-agents and toolsto extract structured information from a document usingonly a local LLM(Mistral-7b-instruct-v0.2).

importlangroidaslrimportlangroid.language_modelsaslm# set up LLMllm_cfg=lm.OpenAIGPTConfig(# or OpenAIAssistant to use Assistant API# any model served via an OpenAI-compatible APIchat_model=lm.OpenAIChatModel.GPT4o,# or, e.g., "ollama/mistral")# use LLM directlymdl=lm.OpenAIGPT(llm_cfg)response=mdl.chat("What is the capital of Ontario?",max_tokens=10)# use LLM in an Agentagent_cfg=lr.ChatAgentConfig(llm=llm_cfg)agent=lr.ChatAgent(agent_cfg)agent.llm_response("What is the capital of China?")response=agent.llm_response("And India?")# maintains conversation state# wrap Agent in a Task to run interactive loop with user (or other agents)task=lr.Task(agent,name="Bot",system_message="You are a helpful assistant")task.run("Hello")# kick off with user saying "Hello"# 2-Agent chat loop: Teacher Agent asks questions to Student Agentteacher_agent=lr.ChatAgent(agent_cfg)teacher_task=lr.Task(teacher_agent,name="Teacher",system_message="""    Ask your student concise numbers questions, and give feedback.    Start with a question.    """)student_agent=lr.ChatAgent(agent_cfg)student_task=lr.Task(student_agent,name="Student",system_message="Concisely answer the teacher's questions.",single_round=True,)teacher_task.add_sub_task(student_task)teacher_task.run()

🔥 Updates/Releases

Click to expand
  • Aug 2025:

    • 0.59.0 Complete Pydantic V2 Migration -5-50x faster validation, modern Python patterns, 100% backward compatible.
  • Jul 2025:

    • 0.58.0 Crawl4AI integration -browser-based web crawling with Playwright for JavaScript-heavy sites, no API key required (thank you @abab-dev!).
    • 0.57.0 HTML Logger for interactive task visualization -self-contained HTML logs with collapsible entries, auto-refresh, and persistent UI state.
  • Jun 2025:

    • 0.56.0TaskTool for delegating tasks to sub-agents -enables agents to spawn sub-agents with specific tools and configurations.
    • 0.55.0 Event-based task termination withdone_sequences -declarative task completion using event patterns.
    • 0.54.0 Portkey AI Gateway support - access 200+ modelsacross providers through unified API with caching, retries, observability.
  • Mar-Apr 2025:

    • 0.53.0 MCP Tools Support.
    • 0.52.0 Multimodal support, i.e. allow PDF, imageinputs to LLM.
    • 0.51.0LLMPdfParser, generalizingGeminiPdfParser to parse documents directly with LLM.
    • 0.50.0 Structure-aware Markdown chunking with chunksenriched by section headers.
    • 0.49.0 Enable easy switch to LiteLLM Proxy-server
    • 0.48.0 Exa Crawler, Markitdown Parser
    • 0.47.0 Support Firecrawl URL scraper/crawler -thanks @abab-dev
    • 0.46.0 Support LangDB LLM Gateway - thanks @MrunmayS.
    • 0.45.0 Markdown parsing withMarker - thanks @abab-dev
    • 0.44.0 Late imports to reduce startup time. Thanks@abab-dev
  • Feb 2025:

    • 0.43.0:GeminiPdfParser for parsing PDF usingGemini LLMs - Thanks @abab-dev.
    • 0.42.0:markitdown parser forpptx,xlsx,xls filesThanks @abab-dev.
    • 0.41.0:pinecone vector-db (Thanks @coretado),Tavily web-search (Thanks @Sozhan308),Exa web-search (Thanks @MuddyHope).
    • 0.40.0:pgvector vector-db. Thanks @abab-dev.
    • 0.39.0:ChatAgentConfig.handle_llm_no_tool forhandling LLM "forgetting" to use a tool.
    • 0.38.0: Gemini embeddings - Thanks @abab-dev)
    • 0.37.0: New PDF Parsers:docling,pymupdf4llm
  • Jan 2025:

    • 0.36.0: Weaviate vector-db support (thanks @abab-dev).
    • 0.35.0: Capture/Stream reasoning content fromReasoning LLMs (e.g. DeepSeek-R1, OpenAI o1) in addition to final answer.
    • 0.34.0: DocChatAgentchunk enrichment to improve retrieval. (collaboration with @dfm88).
    • 0.33.0 Move from Poetry to uv! (thanks @abab-dev).
    • 0.32.0 DeepSeek v3 support.
  • Dec 2024:

    • 0.31.0 Azure OpenAI Embeddings
    • 0.30.0 Llama-cpp embeddings (thanks @Kwigg).
    • 0.29.0 Custom Azure OpenAI Client (thanks@johannestang).
    • 0.28.0ToolMessage:_handler field to overridedefault handler method name inrequest field (thanks @alexagr).
    • 0.27.0 OpenRouter Support.
    • 0.26.0 Update to latest Chainlit.
    • 0.25.0 True Async Methods for agent anduser-response (thanks @alexagr).
  • Nov 2024:

    • 0.24.0:Enables support forAgents with strict JSON schema output format on compatible LLMs and strict mode for the OpenAI tools API.(thanks @nilspalumbo).
    • 0.23.0:support for LLMs (e.g.Qwen2.5-Coder-32b-Instruct) hosted on glhf.chat
    • 0.22.0:Optional parameters to truncate large tool results.
    • 0.21.0 Direct support for Gemini models via OpenAI client instead of using LiteLLM.
    • 0.20.0 Support forArangoDB Knowledge Graphs.
  • Oct 2024:

  • Sep 2024:

    • 0.16.0 Support for OpenAIo1-mini ando1-preview models.
    • 0.15.0 Cerebras API support -- run llama-3.1 models hosted on Cerebras Cloud (very fast inference).
    • 0.14.0DocChatAgent uses Reciprocal Rank Fusion (RRF) to rank chunks retrieved by different methods.
    • 0.12.0run_batch_task new option --stop_on_first_result - allows termination of batch as soon as any task returns a result.
  • Aug 2024:

    • 0.11.0 PolymorphicTask.run(), Task.run_async.
    • 0.10.0 Allow tool handlers to return arbitrary result type, including other tools.
    • 0.9.0 Orchestration Tools, to signal various task statuses, and to pass messages between agents.
    • 0.7.0 OpenAI tools API support, including multi-tools.
  • Jul 2024:

  • Jun 2024:

    • 0.2.0: Improved lineage tracking, granular sub-task configs, and a new tool,RewindTool,that lets an agent "rewind and redo" a past message (and all dependent messages are cleared outthanks to the lineage tracking). Read noteshere.
  • May 2024:

    • Slimmer langroid: All document-parsers (i.e. pdf, doc, docx) and mostvector-databases (except qdrant)are now optional/extra dependencies, which helps reduce build size, scriptstart-up time, and install time. For convenience various grouping of "extras" areprovided, e.g.doc-chat,db (for database-related dependencies). See updatedinstall instructions below and in the docs.
    • Few-shot examples for tools: when defining aToolMessage, previously you were able to include a classmethod namedexamples,and a random example from this list would be used to generate a 1-shot examplefor the LLM. This has been improved so you can now supply a list of exampleswhere each example is either a tool instance, or a tuple of (description,tool instance), where the description is a "thought" that leads the LLM to usethe tool (see example in thedocs). In some scenarios this can improve LLM toolgeneration accuracy. Also, now instead of a random example, ALL examples are used to generate few-shotexamples.
    • Infinite loop detection for task loops of cycle-length <= 10 (configurableinTaskConfig. Only detectsexact loops, rather thanapproximate loops where the entities are saying essentially similar (but not exactly the same) things repeatedly.
    • "@"-addressing: any entity can address any other by name, which can be the nameof an agent's responder ("llm", "user", "agent") or a sub-task name. This is asimpler alternative to theRecipientTool mechanism, with the tradeoff thatsince it's not a tool, there's no way to enforce/remind the LLM to explicitlyspecify an addressee (in scenarios where this is important).
    • Much-Improved Citationgeneration and display when usingDocChatAgent.
    • gpt-4o is now the default LLM throughout; Update tests and examples to workwith this LLM; use tokenizer corresponding to the LLM.
    • gemini 1.5 pro support vialitellm
    • QdrantDB: update to support learned sparse embeddings.
  • Apr 2024:

    • 0.1.236: Support for open LLMs hosted on Groq, e.g. specifychat_model="groq/llama3-8b-8192".Seetutorial.
    • 0.1.235:Task.run(), Task.run_async(), run_batch_tasks havemax_costandmax_tokens params to exit when tokens or cost exceed a limit. The resultChatDocument.metadata now includes astatus field which is a code indicating atask completion reason code. Alsotask.run() etc can be invoked with an explicitsession_id field which is used as a key to look up various settings in Redis cache.Currently only used to look up "kill status" - this allows killing a running task, either bytask.kill()or by the classmethodTask.kill_session(session_id).For example usage, see thetest_task_kill intests/main/test_task.py
  • Mar 2024:

    • 0.1.216: Improvements to allow concurrent runs ofDocChatAgent, see thetest_doc_chat_agent.pyin particular thetest_doc_chat_batch();New task run utility:run_batch_task_genwhere a task generator can be specified, to generate one task per input.
    • 0.1.212: ImagePdfParser: support for extracting text from image-based PDFs.(this meansDocChatAgent will now work with image-pdfs).
    • 0.1.194 - 0.1.211: Misc fixes, improvements, and features:
      • Big enhancement in RAG performance (mainly, recall) due to afix in RelevanceExtractor
      • DocChatAgentcontext-window fixes
      • Anthropic/Claude3 support via Litellm
      • URLLoader: detect file time from header when URL doesn't end with arecognizable suffix like.pdf,.docx, etc.
      • Misc lancedb integration fixes
      • Auto-select embedding config based on whethersentence_transformer module is available.
      • Slim down dependencies, make some heavy ones optional, e.g.unstructured,haystack,chromadb,mkdocs,huggingface-hub,sentence-transformers.
      • Easier top-level imports fromimport langroid as lr
      • Improve JSON detection, esp from weak LLMs
  • Feb 2024:

    • 0.1.193: Support local LLMs using Ollama's new OpenAI-Compatible server:simply specifychat_model="ollama/mistral". Seerelease notes.
    • 0.1.183: Added Chainlit support viacallbacks.Seeexamples.
  • Jan 2024:

    • 0.1.175
      • Neo4jChatAgent to chat with a neo4j knowledge-graph.(Thanks toMohannad!). The agent uses tools to query the Neo4j schema and translate user queries to Cypher queries,and the tool handler executes these queries, returning them to the LLM to composea natural language response (analogous to howSQLChatAgent works).See examplescript using this Agent to answer questions about Python pkg dependencies.
      • Support for.doc file parsing (in addition to.docx)
      • Specify optionalformatter paraminOpenAIGPTConfig to ensure accurate chat formatting for local LLMs.
    • 0.1.157:DocChatAgentConfighas a new param:add_fields_to_content, to specify additional document fields to insert intothe maincontent field, to help improve retrieval.
    • 0.1.156: New Task control signalsPASS_TO, SEND_TO; VectorStore: Compute Pandas expression on documents; LanceRAGTaskCreator creates 3-agent RAG system with Query Planner, Critic and RAG Agent.
  • Dec 2023:

    • 0.1.154: (For details see release notes of0.1.149and0.1.154).
      • DocChatAgent: Ingest Pandas dataframes and filtering.
      • LanceDocChatAgent leveragesLanceDB vector-db for efficient vector searchand full-text search and filtering.
      • Improved task and multi-agent control mechanisms
      • LanceRAGTaskCreator to create a 2-agent system consisting of aLanceFilterAgent thatdecides a filter and rephrase query to send to a RAG agent.
    • 0.1.141:API Simplifications to reduce boilerplate:auto-select an available OpenAI model (preferring gpt-4o), simplifies defaults.SimplerTask initialization with defaultChatAgent.
  • Nov 2023:

  • Oct 2023:

    • 0.1.107:DocChatAgent re-rankers:rank_with_diversity,rank_to_periphery (lost in middle).
    • 0.1.102:DocChatAgentConfig.n_neighbor_chunks > 0 allows returning context chunks around match.
    • 0.1.101:DocChatAgent usesRelevanceExtractorAgent to havethe LLM extract relevant portions of a chunk usingsentence-numbering, resulting in huge speed up and cost reductioncompared to the naive "sentence-parroting" approach (writing out fullsentences out relevant whole sentences) whichLangChain uses in theirLLMChainExtractor.
    • 0.1.100: API update: all of Langroid is accessible with a single import, i.e.import langroid as lr. See the documentation for usage.
    • 0.1.99: Convenience batch functions to run tasks, agent methods on a list of inputs concurrently in async mode. See examples intest_batch.py.
    • 0.1.95: Added support forMomento Serverless Vector Index
    • 0.1.94: Added support forLanceDB vector-store -- allows vector, Full-text, SQL search.
    • 0.1.84: AddedLiteLLM, so now Langroid can be used with over 100 LLM providers (remote or local)!See guidehere.
  • Sep 2023:

    • 0.1.78: Async versions of several Task, Agent and LLM methods;Nested Pydantic classes are now supported for LLM Function-calling, Tools, Structured Output.
    • 0.1.76: DocChatAgent: support for loadingdocx files (preliminary).
    • 0.1.72: Many improvements to DocChatAgent: better embedding model,hybrid search to improve retrieval, better pdf parsing, re-ranking retrieved results with cross-encoders.
    • Use with local LLama Models: see tutorialhere
    • Langroid Blog/Newsletter Launched!: First post ishere -- Please subscribe to stay updated.
    • 0.1.56: Support Azure OpenAI.
    • 0.1.55: ImprovedSQLChatAgent that efficiently retrieves relevant schema info when translating natural language to SQL.
  • Aug 2023:

  • July 2023:

🚀 Demo

Suppose you want to extract structured information about the key termsof a commercial lease document. You can easily do this with Langroid using a two-agent system,as we show in thelangroid-examples repo.(Seethis scriptfor a version with the same functionality using a local Mistral-7b model.)The demo showcases just a few of the many features of Langroid, such as:

  • Multi-agent collaboration:LeaseExtractor is in charge of the task, and its LLM (GPT4) generates questionsto be answered by theDocAgent.
  • Retrieval augmented question-answering, withsource-citation:DocAgent LLM (GPT4) uses retrieval from a vector-store toanswer theLeaseExtractor's questions, cites the specific excerpt supporting the answer.
  • Function-calling (also known as tool/plugin): When it has all the information itneeds, theLeaseExtractor LLM presents the information in a structuredformat using a Function-call.

Here is what it looks like in action(a pausable mp4 video ishere).

Demo

⚡ Highlights

(For a more up-to-date list see theUpdates/Releasessection above)

  • Agents as first-class citizens: TheAgent class encapsulates LLM conversation state,and optionally a vector-store and tools. Agents are a core abstraction in Langroid;Agents act asmessage transformers, and by default provide 3responder methods, one corresponding to each entity: LLM, Agent, User.
  • Tasks: ATask class wraps an Agent, and gives the agent instructions (or roles, or goals),manages iteration over an Agent's responder methods,and orchestrates multi-agent interactions via hierarchical, recursivetask-delegation. TheTask.run() method has the sametype-signature as an Agent's responder's methods, and this is key to howa task of an agent can delegate to other sub-tasks: from the point of view of a Task,sub-tasks are simply additional responders, to be used in a round-robin fashionafter the agent's own responders.
  • Modularity, Reusability, Loose coupling: TheAgent andTask abstractions allow users to designAgents with specific skills, wrap them in Tasks, and combine tasks in a flexible way.
  • LLM Support: Langroid supports OpenAI LLMs as well as LLMs from hundreds ofproviders (local/open orremote/commercial) via proxy libraries and local model serverssuch asollama,oobabooga,LiteLLM that in effect mimic the OpenAI API. See thesupported LLMs.
  • Caching of LLM responses: Langroid supportsRedis to cache LLM responses.
  • Vector-stores:Qdrant,Chroma, LanceDB, Pinecone, PostgresDB (PGVector), Weaviate are currently supported.Vector stores allow for Retrieval-Augmented-Generation (RAG).
  • Grounding and source-citation: Access to external documents via vector-storesallows for grounding and source-citation.
  • Observability, Logging, Lineage: Langroid generates detailed logs of multi-agent interactions andmaintains provenance/lineage of messages, so that you can trace backthe origin of a message.
  • Tools/Plugins/Function-calling:Langroid supports OpenAI'sfunction calling, aswell as an equivalentToolMessage mechanism which works withany LLM, not just OpenAI's.Function calling and tools have the same developer-facing interface, implementedusingPydantic,which makes it very easy to define tools/functions and enable agentsto use them. Benefits of using Pydantic are that you never have to writecomplex JSON specs for function calling, and when the LLMhallucinates malformed JSON, the Pydantic error message is sent back tothe LLM so it can fix it.

⚙️ Installation and Setup

Installlangroid

Langroid requires Python 3.11+. We recommend using a virtual environment.Usepip to install a bare-bones slim version oflangroid (from PyPi) to your virtualenvironment:

pip install langroid

The core Langroid package lets you use OpenAI Embeddings models via their API.If you instead want to use thesentence-transformers embedding models from HuggingFace,install Langroid like this:

pip install"langroid[hf-embeddings]"

For many practical scenarios, you may need additional optional dependencies:

  • To use various document-parsers, install langroid with thedoc-chat extra:
    pip install"langroid[doc-chat]"
  • For "chat with databases", use thedb extra:
    pip install"langroid[db]"
  • You can specify multiple extras by separating them with commas, e.g.:
    pip install"langroid[doc-chat,db]"
  • To simply installall optional dependencies, use theall extra (but note that this will result in longer load/startup times and a larger install size):
    pip install"langroid[all]"
Optional Installs for using SQL Chat with a PostgreSQL DB

If you are usingSQLChatAgent(e.g. the scriptexamples/data-qa/sql-chat/sql_chat.py),with a postgres db, you will need to:

  • Install PostgreSQL dev libraries for your platform, e.g.
    • sudo apt-get install libpq-dev on Ubuntu,
    • brew install postgresql on Mac, etc.
  • Install langroid with the postgres extra, e.g.pip install langroid[postgres]orpoetry add "langroid[postgres]" orpoetry install -E postgres,(or the correspondinguv versions, e.g.uv add "langroid[postgres]"oruv pip install langroid[postgres]).If this gives you an error, trypip install psycopg2-binary in your virtualenv.

📝 If you get strange errors involvingmysqlclient, try doingpip uninstall mysqlclient followed bypip install mysqlclient.

Claude Code Plugin (Optional)

This plugin provides two skills:

  • langroid:patterns - Your Claude Code agent can leverage this skill to produceLangroid multi-agent code using proper design patterns.
  • langroid:add-pattern - The agent can use this skill to record new patterns itlearns, for future reference, either autonomously or when prompted by the user.

Step 1: Add the Langroid marketplace

From terminal:

claude plugin marketplace add langroid/langroid

Or within Claude Code:

/plugin marketplace add langroid/langroid

Step 2: Install the Langroid plugin

From terminal:

claude plugin install langroid@langroid

Or within Claude Code:

/plugin install langroid@langroid

Once installed, simply ask your Claude Code agent to implement Langroid patterns innatural language, e.g.,

set up a Langroid agent so it uses the EditTool, and wrap it in a task that ends as soon as the tool is generated

and it will automatically use thelangroid:patterns skill to follow the right design pattern.

You can also ask Claude Code to record a new pattern when you discover one, e.g.,

record this as a new Langroid pattern for setting up MCP tools

Set up environment variables (API keys, etc)

To get started, all you need is an OpenAI API Key.If you don't have one, seethis OpenAI Page.(Note that while this is the simplest way to get started, Langroid works with practically any LLM, not just those from OpenAI.See the guides to usingOpen/Local LLMs,and othernon-OpenAI proprietary LLMs.)

In the root of the repo, copy the.env-template file to a new file.env:

cp .env-template .env

Then insert your OpenAI API Key.Your.env file should look like this (the organization is optionalbut may be required in some scenarios).

OPENAI_API_KEY=your-key-here-without-quotesOPENAI_ORGANIZATION=optionally-your-organization-id

Alternatively, you can set this as an environment variable in your shell(you will need to do this every time you open a new shell):

export OPENAI_API_KEY=your-key-here-without-quotes
Optional Setup Instructions (click to expand)

All of the following environment variable settings are optional, and some are only neededto use specific features (as noted below).

  • Qdrant Vector Store API Key, URL. This is only required if you want to use Qdrant cloud.AlternativelyChroma orLanceDB are also currently supported.We use the local-storage version of Chroma, so there is no need for an API key.
  • Redis Password, host, port: This is optional, and only needed to cache LLM API responsesusing Redis Cloud. Redisoffers a free 30MB Redis accountwhich is more than sufficient to try out Langroid and even beyond.If you don't set up these, Langroid will use a pure-pythonRedis in-memory cache via theFakeredis library.
  • Momento Serverless Caching of LLM API responses (as an alternative to Redis).To use Momento instead of Redis:
    • enter your Momento Token in the.env file, as the value ofMOMENTO_AUTH_TOKEN (see example file below),
    • in the.env file setCACHE_TYPE=momento (instead ofCACHE_TYPE=redis which is the default).
  • GitHub Personal Access Token (required for apps that need to analyze gitrepos; token-based API calls are less rate-limited). See thisGitHub page.
  • Google Custom Search API Credentials: Only needed to enable an Agent to use theGoogleSearchTool.To use Google Search as an LLM Tool/Plugin/function-call,you'll need to set upa Google API key,thensetup a Google Custom Search Engine (CSE) and get the CSE ID.(Documentation for these can be challenging, we suggest asking GPT4 for a step-by-step guide.)After obtaining these credentials, store them as values ofGOOGLE_API_KEY andGOOGLE_CSE_ID in your.env file.Full documentation on using this (and other such "stateless" tools) is coming soon, butin the meantime take a peek at thischat example, whichshows how you can easily equip an Agent with aGoogleSearchtool.

If you add all of these optional variables, your.env file should look like this:

OPENAI_API_KEY=your-key-here-without-quotesGITHUB_ACCESS_TOKEN=your-personal-access-token-no-quotesCACHE_TYPE=redis# or momentoREDIS_PASSWORD=your-redis-password-no-quotesREDIS_HOST=your-redis-hostname-no-quotesREDIS_PORT=your-redis-port-no-quotesMOMENTO_AUTH_TOKEN=your-momento-token-no-quotes# instead of REDIS* variablesQDRANT_API_KEY=your-keyQDRANT_API_URL=https://your.url.here:6333# note port number must be includedGOOGLE_API_KEY=your-keyGOOGLE_CSE_ID=your-cse-id
Optional setup instructions for Microsoft Azure OpenAI(click to expand)

When using Azure OpenAI, additional environment variables are required in the.env file.This pageMicrosoft Azure OpenAIprovides more information, and you can set each environment variable as follows:

  • AZURE_OPENAI_API_KEY, from the value ofAPI_KEY
  • AZURE_OPENAI_API_BASE from the value ofENDPOINT, typically looks likehttps://your.domain.azure.com.
  • ForAZURE_OPENAI_API_VERSION, you can use the default value in.env-template, and latest version can be foundhere
  • AZURE_OPENAI_DEPLOYMENT_NAME is the name of the deployed model, which is defined by the user during the model setup
  • AZURE_OPENAI_MODEL_NAME Azure OpenAI allows specific model names when you select the model for your deployment. You need to put precisly the exact model name that was selected. For example, GPT-4 (should begpt-4-32k orgpt-4).
  • AZURE_OPENAI_MODEL_VERSION is required ifAZURE_OPENAI_MODEL_NAME = gpt=4, which will assist Langroid to determine the cost of the model

🐳 Docker Instructions

We provide a containerized version of thelangroid-examplesrepository via thisDocker Image.All you need to do is set up environment variables in the.env file.Please follow these steps to setup the container:

# get the .env file template from `langroid` repowget -O .env https://raw.githubusercontent.com/langroid/langroid/main/.env-template# Edit the .env file with your favorite editor (here nano), and remove any un-used settings. E.g. there are "dummy" values like "your-redis-port" etc -- if you are not using them, you MUST remove them.nano .env# launch the container (the appropriate image for your architecture will be pulled automatically)docker run -it --rm  -v ./.env:/langroid/.env langroid/langroid:latest# Use this command to run any of the scripts in the `examples` directorypython examples/<Path/To/Example.py>

🎉 Usage Examples

These are quick teasers to give a glimpse of what you can do with Langroidand how your code would look.

⚠️ The code snippets below are intended to give a flavor of the codeand they arenot complete runnable examples! For that we encourage you toconsult thelangroid-examplesrepository.

ℹ️The various LLM prompts and instructions in Langroidhave been tested to work well with GPT-4 (and to some extent GPT-4o).Switching to other LLMs (local/open and proprietary) is easy (see guides mentioned above),and may suffice for some applications, but in general you may see inferior resultsunless you adjust the prompts and/or the multi-agent setup.

📖 Also see theGetting Started Guidefor a detailed tutorial.

Click to expand any of the code examples below.All of these can be run in a Colab notebook:Open in Colab

Direct interaction with LLM
importlangroid.language_modelsaslmmdl=lm.OpenAIGPT(lm.OpenAIGPTConfig(chat_model=lm.OpenAIChatModel.GPT4o,# or, e.g.  "ollama/qwen2.5"    ),)messages= [lm.LLMMessage(content="You are a helpful assistant",role=lm.Role.SYSTEM),lm.LLMMessage(content="What is the capital of Ontario?",role=lm.Role.USER),]response=mdl.chat(messages,max_tokens=200)print(response.message)

See the guides to use(local/open LLMs orremote/commercial LLMs).

Interaction with non-OpenAI LLM (local or remote)Local model: if model is served at `http://localhost:8000`:
cfg=lm.OpenAIGPTConfig(chat_model="local/localhost:8000",chat_context_length=4096)mdl=lm.OpenAIGPT(cfg)# now interact with it as above, or create an Agent + Task as shown below.
Define an agent, set up a task, and run it
importlangroidaslragent=lr.ChatAgent()# get response from agent's LLM, and put this in an interactive loop...# answer = agent.llm_response("What is the capital of Ontario?")# ... OR instead, set up a task (which has a built-in loop) and run ittask=lr.Task(agent,name="Bot")task.run()# ... a loop seeking response from LLM or User at each turn
Three communicating agents

A toy numbers game, where when given a numbern:

  • repeater_task's LLM simply returnsn,
  • even_task's LLM returnsn/2 ifn is even, else says "DO-NOT-KNOW"
  • odd_task's LLM returns3*n+1 ifn is odd, else says "DO-NOT-KNOW"

Each of theseTasks automatically configures a defaultChatAgent.

importlangroidaslrfromlangroid.utils.constantsimportNO_ANSWERrepeater_task=lr.Task(name="Repeater",system_message="""    Your job is to repeat whatever number you receive.    """,llm_delegate=True,# LLM takes charge of tasksingle_round=False, )even_task=lr.Task(name="EvenHandler",system_message=f"""    You will be given a number.    If it is even, divide by 2 and say the result, nothing else.    If it is odd, say{NO_ANSWER}    """,single_round=True,# task done after 1 step() with valid response)odd_task=lr.Task(name="OddHandler",system_message=f"""    You will be given a number n.    If it is odd, return (n*3+1), say nothing else.    If it is even, say{NO_ANSWER}    """,single_round=True,# task done after 1 step() with valid response)

Then add theeven_task andodd_task as sub-tasks ofrepeater_task,and run therepeater_task, kicking it off with a number as input:

repeater_task.add_sub_task([even_task,odd_task])repeater_task.run("3")
Simple Tool/Function-calling example

Langroid leverages Pydantic to support OpenAI'sFunction-calling APIas well as its own native tools. The benefits are that you don't have to writeany JSON to specify the schema, and also if the LLM hallucinates a malformedtool syntax, Langroid sends the Pydantic validation error (suitably sanitized)to the LLM so it can fix it!

Simple example: Say the agent has a secret list of numbers,and we want the LLM to find the smallest number in the list.We want to give the LLM aprobe tool/function which takes asingle numbern as argument. The tool handler method in the agentreturns how many numbers in its list are at mostn.

First define the tool using Langroid'sToolMessage class:

importlangroidaslrclassProbeTool(lr.agent.ToolMessage):request:str="probe"# specifies which agent method handles this toolpurpose:str="""        To find how many numbers in my list are less than or equal to        the <number> you specify.        """# description used to instruct the LLM on when/how to use the toolnumber:int# required argument to the tool

Then define aSpyGameAgent as a subclass ofChatAgent,with a methodprobe that handles this tool:

classSpyGameAgent(lr.ChatAgent):def__init__(self,config:lr.ChatAgentConfig):super().__init__(config)self.numbers= [3,4,8,11,15,25,40,80,90]defprobe(self,msg:ProbeTool)->str:# return how many numbers in self.numbers are less or equal to msg.numberreturnstr(len([nforninself.numbersifn<=msg.number]))

We then instantiate the agent and enable it to use and respond to the tool:

spy_game_agent=SpyGameAgent(lr.ChatAgentConfig(name="Spy",vecdb=None,use_tools=False,#  don't use Langroid native tooluse_functions_api=True,# use OpenAI function-call API    ))spy_game_agent.enable_message(ProbeTool)

For a full working example see thechat-agent-tool.pyscript in thelangroid-examples repo.

Tool/Function-calling to extract structured information from text

Suppose you want an agent to extractthe key terms of a lease, from a lease document, as a nested JSON structure.First define the desired structure via Pydantic models:

frompydanticimportBaseModelclassLeasePeriod(BaseModel):start_date:strend_date:strclassLeaseFinancials(BaseModel):monthly_rent:strdeposit:strclassLease(BaseModel):period:LeasePeriodfinancials:LeaseFinancialsaddress:str

Then define theLeaseMessage tool as a subclass of Langroid'sToolMessage.Note the tool has a required argumentterms of typeLease:

importlangroidaslrclassLeaseMessage(lr.agent.ToolMessage):request:str="lease_info"purpose:str="""        Collect information about a Commercial Lease.        """terms:Lease

Then define aLeaseExtractorAgent with a methodlease_info that handles this tool,instantiate the agent, and enable it to use and respond to this tool:

classLeaseExtractorAgent(lr.ChatAgent):deflease_info(self,message:LeaseMessage)->str:print(f"""        DONE! Successfully extracted Lease Info:{message.terms}        """        )returnjson.dumps(message.terms.dict())lease_extractor_agent=LeaseExtractorAgent()lease_extractor_agent.enable_message(LeaseMessage)

See thechat_multi_extract.pyscript in thelangroid-examples repo for a full working example.

Chat with documents (file paths, URLs, etc)

Langroid provides a specialized agent classDocChatAgent for this purpose.It incorporates document sharding, embedding, storage in a vector-DB,and retrieval-augmented query-answer generation.Using this class to chat with a collection of documents is easy.First create aDocChatAgentConfig instance, with adoc_paths field that specifies the documents to chat with.

importlangroidaslrfromlangroid.agent.specialimportDocChatAgentConfig,DocChatAgentconfig=DocChatAgentConfig(doc_paths= ["https://en.wikipedia.org/wiki/Language_model","https://en.wikipedia.org/wiki/N-gram_language_model","/path/to/my/notes-on-language-models.txt",  ],vecdb=lr.vector_store.QdrantDBConfig(),)

Then instantiate theDocChatAgent (this ingests the docs into the vector-store):

agent=DocChatAgent(config)

Then we can either ask the agent one-off questions,

agent.llm_response("What is a language model?")

or wrap it in aTask and run an interactive loop with the user:

task=lr.Task(agent)task.run()

See full working scripts in thedocqafolder of thelangroid-examples repo.

🔥 Chat with tabular data (file paths, URLs, dataframes)

Using Langroid you can set up aTableChatAgent with a dataset (file path, URL or dataframe),and query it. The Agent's LLM generates Pandas code to answer the query,via function-calling (or tool/plugin), and the Agent's function-handling methodexecutes the code and returns the answer.

Here is how you can do this:

importlangroidaslrfromlangroid.agent.specialimportTableChatAgent,TableChatAgentConfig

Set up aTableChatAgent for a data file, URL or dataframe(Ensure the data table has a header row; the delimiter/separator is auto-detected):

dataset="https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv"# or dataset = "/path/to/my/data.csv"# or dataset = pd.read_csv("/path/to/my/data.csv")agent=TableChatAgent(config=TableChatAgentConfig(data=dataset,    ))

Set up a task, and ask one-off questions like this:

task=lr.Task(agent,name="DataAssistant",default_human_response="",# to avoid waiting for user input)result=task.run("What is the average alcohol content of wines with a quality rating above 7?",turns=2# return after user question, LLM fun-call/tool response, Agent code-exec result)print(result.content)

Or alternatively, set up a task and run it in an interactive loop with the user:

task=lr.Task(agent,name="DataAssistant")task.run()

For a full working example see thetable_chat.pyscript in thelangroid-examples repo.


❤️ Thank you to oursupporters

If you like this project, please give it a star ⭐ and 📢 spread the word in your network or social media:

Share on TwitterShare on LinkedInShare on Hacker NewsShare on Reddit

Your support will help build Langroid's momentum and community.

Langroid Co-Founders

  • Prasad Chalasani (IIT BTech/CS, CMU PhD/ML; Independent ML Consultant)
  • Somesh Jha (IIT BTech/CS, CMU PhD/CS; Professor of CS, U Wisc at Madison)

Sponsor this project

 

Packages

No packages published

Languages


[8]ページ先頭

©2009-2026 Movatter.jp