Movatterモバイル変換

Skip to content

#

webscraping

Here are 9,884 public repositories matching this topic...

Language:All

Filter by language

All9,884 Python4,851 Jupyter Notebook2,267 JavaScript718 HTML424 TypeScript266 R181 Java172 Go114 C#102 PHP66

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

firecrawl

firecrawl /firecrawl

🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

markdown crawler scraper ai html-to-markdown web-crawler scraping web-scraper web-scraping data-extraction webscraping web-data-extraction ai-agents web-search ai-search web-data llm ai-crawler ai-scraping

UpdatedNov 29, 2025
TypeScript

huginn /huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

notifications agent rss scraper automation twitter monitoring huginn feed feedgenerator webscraping twitter-streaming

UpdatedNov 29, 2025
Ruby

gpt-researcher

assafelovic /gpt-researcher

An LLM agent that conducts deep research (local and web) on any given topic and generates a long report with citations.

python search agent automation research ai mcp webscraping llms mcp-server deepresearch

UpdatedNov 15, 2025
Python

ScrapeGraphAI /Scrapegraph-ai

Python scraper based on AI

markdown crawler web-crawler scraping web-scraper web-scraping data-extraction webscraping web-data-extraction web-search ai-search rag automated-scraper web-data scraping-python web-crawlers llm ai-crawler large-language-model ai-scraping

UpdatedNov 26, 2025
Python

getmaxun /maxun

Turn any website into clean, contextualized data pipelines for your workflows

api scraper automation browser web-scraper self-hosted web-scraping data-extraction webscraping agents hacktoberfest browser-automation no-code web-automation rpa robotic-process-automation nocode playwright hacktoberfest-accepted no-code-web-scraper

UpdatedNov 29, 2025
TypeScript

pystardust /ani-cli

A cli tool to browse and play anime

windows linux shell cli mac terminal anime fzf posix rofi termux webscraping syncplay steamdeck

UpdatedSep 15, 2025
Shell

Scrapling

D4Vinci /Scrapling

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

python crawler data automation ai mcp scraping crawling web-scraper web-scraping selectors xpath data-extraction stealth webscraping crawling-python playwright web-scraping-python ai-scraping mcp-server

UpdatedNov 26, 2025
Python

lorien /awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

crawler spider scraping crawling web-scraping captcha-recaptcha webscraping crawling-framework scraping-framework captcha-bypass scraping-tool crawling-tool scraping-python crawling-python

UpdatedOct 13, 2025
Makefile

autoscraper

alirezamika /autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

python crawler machine-learning scraper automation ai scraping artificial-intelligence web-scraping scrape webscraping webautomation

UpdatedJun 9, 2025
Python

niespodd /browser-fingerprinting

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?

bot crawler scraper automation recaptcha web spider detection chromium chromedriver fingerprinting browser-fingerprinting stealth webscraping bot-detection chromium-browser puppeteer

UpdatedJul 17, 2024
JavaScript

jaypyles /Scraperr

Self-hosted webscraper.

python docker kubernetes opensource helm scraping webscraper web-scraper self-hosted web-scraping web-scrapers webscraping playwright

UpdatedOct 12, 2025
TypeScript

daijro /camoufox

🦊 Anti-detect browser

firefox networking scraping fingerprint webscraping playwright antidetect antidetect-browser

UpdatedMar 15, 2025
C++

scrapoxy /scrapoxy

Scrapoxy is a super proxies manager that orchestrates all your proxies into one place, rather than spreading management across multiple scrapers. It manages IP rotation and fingerprinting, and smartly routes traffic to avoid bans.

proxies webscraping blacklisting antibot

UpdatedAug 12, 2025

anaskhan96 /soup

Web Scraper in Go, similar to BeautifulSoup

go golang webscraper web-scraper beautifulsoup webscraping html-node

UpdatedNov 2, 2023
Go

itsOwen /CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

scraper web-scraper openai webscraping gemini-api llm llm-scraper ai-scraping

UpdatedNov 20, 2025
Python

patchright

Kaliiiiiiiiii-Vinyzu /patchright

Undetected version of the Playwright testing and automation library.

bot chrome automation webdriver browser bots chromium cloudflare web-scraping chromedriver stealth webscraping botting web-automation webautomation undetected undetectable cloudflare-bypass playwright

UpdatedNov 26, 2025
JavaScript

reworkd /tarsier

Vision utilities for web interaction agents 👀

python ocr selenium webscraping pypi-package playwright llms gpt4v

UpdatedNov 25, 2024
Jupyter Notebook

TheWebScrapingClub /webscraping-from-0-to-hero

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

python scrapy-spider scrapy webscraping scrapysplash playwright

UpdatedMay 27, 2024

requests-cache

requests-cache /requests-cache

Persistent HTTP cache for python requests

redis http performance web mongodb sqlite cache dynamodb requests webscraping

UpdatedNov 27, 2025
Python

m8sec /CrossLinked

LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping

osint python3 enumeration webscraping pentest-scripts linkedin-scraper pentest-tool username-generator

UpdatedNov 26, 2024
Python

Improve this page

Add a description, image, and links to thewebscraping topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thewebscraping topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp