Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Enhance LLM agents with rich tool APIs

License

NotificationsYou must be signed in to change notification settings

InternLM/agentlego

Repository files navigation

Open in OpenXLabdocsPyPIlicense

English |简体中文

Introduction

AgentLego is an open-source library of versatile tool APIs to extend and enhance large language model (LLM) based agents, with the following highlight features:

  • Rich set of tools for multimodal extensions of LLM agents including visual perception, image generation and editing, speech processing and visual-language reasoning, etc.
  • Flexible tool interface that allows users to easily extend custom tools with arbitrary types of arguments and outputs.
  • Easy integration with LLM-based agent frameworks likeLangChain,Transformers Agents,Lagent.
  • Support tool serving and remote accessing, which is especially useful for tools with heavy ML models (e.g. ViT) or special environment requirements (e.g. GPU and CUDA).
AgentLego._720p.mp4

Quick Starts

Installation

Install the AgentLego package

pip install agentlego

Install tool-specific dependencies

Some tools requires extra packages, please check the readme file of the tool, and confirm all requirements aresatisfied.

For example, if we want to use theImageDescription tool. We need to check theSet up section ofreadme and install the requirements.

pip install -U openmimmim install -U mmpretrain

Use tools directly

fromagentlegoimportlist_tools,load_toolprint(list_tools())# list tools in AgentLegoimage_caption_tool=load_tool('ImageDescription',device='cuda')print(image_caption_tool.description)image='./examples/demo.png'caption=image_caption_tool(image)

Integrated into agent frameworks

Supported Tools

General ability

Speech related

Image-processing related

  • ImageDescription: Describe the input image.
  • OCR: Recognize the text from a photo.
  • VQA: Answer the question according to the image.
  • HumanBodyPose: Estimate the pose or keypoints of human in an image.
  • HumanFaceLandmark: Estimate the landmark or keypoints of human faces in an image.
  • ImageToCanny: Extract the edge image from an image.
  • ImageToDepth: Generate the depth image of an image.
  • ImageToScribble: Generate a sketch scribble of an image.
  • ObjectDetection: Detect all objects in the image.
  • TextToBbox: Detect specific objects described by the given text in the image.
  • Segment Anything series

AIGC related

Licence

This project is released under theApache 2.0 license. Users should also ensure compliance with the licenses governing the models used in this project.


[8]ページ先頭

©2009-2025 Movatter.jp