web-scraping
Here are 10,220 public repositories matching this topic...
Language:All
Sort:Most stars
🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data
- Updated
Feb 20, 2026 - TypeScript
Scrapy, a fast high-level web crawling & scraping framework for Python.
- Updated
Feb 20, 2026 - Python
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.
- Updated
Feb 19, 2026 - JavaScript
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!
- Updated
Feb 20, 2026 - Python
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- Updated
Feb 20, 2026 - TypeScript
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。
- Updated
Oct 12, 2025 - Python
✨ The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes ✨
- Updated
Feb 20, 2026 - TypeScript
Python APIs for web automation, testing, and bypassing bot-detection with ease.
- Updated
Feb 20, 2026 - Python
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
- Updated
Feb 20, 2026 - Python
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
- Updated
Feb 18, 2026 - Python
Lighter web automation with Python
- Updated
Feb 4, 2026 - Python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
- Updated
Feb 20, 2026 - Python
List of libraries, tools and APIs for web scraping and data processing.
- Updated
Jan 13, 2026 - Makefile
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
- Updated
Jun 9, 2025 - Python
A Chrome DevTools Protocol driver for web automation and scraping.
- Updated
Feb 17, 2026 - Go
Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
- Updated
Feb 19, 2026 - Python
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.
- Updated
Feb 20, 2026 - JavaScript
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- Updated
Sep 12, 2025 - Python
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
- Updated
Feb 18, 2026 - Python
Improve this page
Add a description, image, and links to theweb-scraping topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with theweb-scraping topic, visit your repo's landing page and select "manage topics."