Repository files navigation

😽News |🐈Setup |🧶Comparison |🐈‍⬛Artifacts |📝Citation |😻Acknowledgement

• Dec 2nd, 2024: We integrated Agentless with Claude 3.5 Sonnet to achieve 40.7% and 50.8% solve rate on SWE-bench lite and verified
• Oct 28th, 2024: We just released OpenAutoCoder-Agentless 1.5!
• July 1st, 2024: We just released OpenAutoCoder-Agentless 1.0!Agentless currently is the best open-source approach on SWE-bench lite with 82 fixes (27.3%) and costing on average $0.34 per issue.

Agentless is anagentless approach to automatically solve software development problems. To solve each issue,Agentless follows a simple three phase process: localization, repair, and patch validation.

• 🙀Localization: Agentless employs a hierarchical process to first localize the fault to specific files, then to relevant classes or functions, and finally to fine-grained edit locations
• 😼Repair: Agentless takes the edit locations and samples multiple candidate patches per bug in a simple diff format
• 😸Patch Validation: Agentless selects the regression tests to run and generates additional reproduction test to reproduce the original error. Using the test results, Agentless re-ranks all remaining patches to selects one to submit

First create the environment

git clone https://github.com/OpenAutoCoder/Agentless.gitcd Agentlessconda create -n agentless python=3.11 conda activate agentlesspip install -r requirements.txtexport PYTHONPATH=$PYTHONPATH:$(pwd)

# for contribution, please install the pre-commit hook.pre-commit install# this allows a more standardized code style

Then export your OpenAI API key

export OPENAI_API_KEY={key_here}

Now you are ready to runAgentless on the problems in SWE-bench!

Below shows the comparison graph betweenAgentless and the best open-source agent-based approaches on SWE-bench lite

You can download the complete artifacts ofAgentless in ourv1.5.0 release:

• 🐈‍⬛ agentless_swebench_lite: complete Agentless run on SWE-bench Lite
• 🐈‍⬛ agentless_swebench_verified: complete Agentless run on SWE-bench Verified
• 🐈‍⬛ swebench_repo_structure: preprocessed structure information for each SWE-Bench problem

You can also checkoutclassification/ folder to obtain our manual classifications of SWE-bench-lite as well as our filtered SWE-bench-lite-S problems.

@article{agentless,author    ={Xia, Chunqiu Steven and Deng, Yinlin and Dunn, Soren and Zhang, Lingming},title     ={Agentless: Demystifying LLM-based Software Engineering Agents},year      ={2024},journal   ={arXiv preprint},}

• SWE-bench
• Aider
• SWE-bench-docker