web-scraping
Here are 6,240 public repositories matching this topic...
Language:All
Sort:Most stars
Scrapy, a fast high-level web crawling & scraping framework for Python.
- Updated
Mar 20, 2025 - Python
The best and simplest free open source web page change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
- Updated
Mar 20, 2025 - Python
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- Updated
Mar 20, 2025 - TypeScript
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。
- Updated
Mar 16, 2025 - Python
Open-source no-code web data extraction platform. Turn websites to APIs & spreadsheets with no-code robots in minutes.
- Updated
Mar 20, 2025 - TypeScript
Python APIs for web automation, testing, and bypassing bot-detection.
- Updated
Mar 20, 2025 - Python
Lighter web automation with Python
- Updated
Feb 20, 2025 - Python
List of libraries, tools and APIs for web scraping and data processing.
- Updated
Dec 27, 2024 - Makefile
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
- Updated
Oct 12, 2024 - Python
A Chrome DevTools Protocol driver for web automation and scraping.
- Updated
Dec 7, 2024 - Go
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
- Updated
Mar 20, 2025 - Python
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- Updated
Mar 17, 2025 - Python
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
- Updated
Mar 16, 2025 - PHP
Snoop — инструмент разведки на основе открытых данных (OSINT world)
- Updated
Mar 19, 2025 - Python
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
- Updated
Mar 13, 2025 - Python
🕷️ An undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy again!
- Updated
Mar 18, 2025 - Python
The Python Code Tutorials
- Updated
Nov 11, 2024 - Jupyter Notebook
Web Scraping Framework
- Updated
Mar 12, 2024 - Python
Learn Python for the next 30 (or so) Days.
- Updated
Feb 27, 2024 - HTML
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
- Updated
Feb 10, 2025 - Python
Improve this page
Add a description, image, and links to theweb-scraping topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with theweb-scraping topic, visit your repo's landing page and select "manage topics."