Movatterモバイル変換

Skip to content

#

webscraping

Here are 9,564 public repositories matching this topic...

Language:All

Filter by language

All9,564 Python4,684 Jupyter Notebook2,195 JavaScript707 HTML413 TypeScript240 R179 Java172 Go111 C#97 PHP65

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

huginn /huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

notifications agent rss scraper automation twitter monitoring huginn feed feedgenerator webscraping twitter-streaming

UpdatedJul 17, 2025
Ruby

mendableai /firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

markdown crawler data scraper ai html-to-markdown web-crawler scraping webscraping rag llm ai-scraping

UpdatedJul 18, 2025
TypeScript

gpt-researcher

assafelovic /gpt-researcher

LLM based autonomous agent that conducts deep local and web research on any topic and generates a long report with citations.

python search agent automation research ai mcp webscraping llms mcp-server deepresearch

UpdatedJul 15, 2025
Python

getmaxun /maxun

🔥 Open-source no code web data extraction platform. Instantly turn any website into API or spreadsheet 🔥

api scraper automation browser web-scraper self-hosted web-scraping data-extraction webscraping agents browser-automation no-code web-automation rpa robotic-process-automation playwright website-to-api no-code-web-scraper

UpdatedJul 18, 2025
TypeScript

pystardust /ani-cli

A cli tool to browse and play anime

windows linux shell cli mac terminal anime fzf posix rofi termux webscraping syncplay steamdeck

UpdatedJul 14, 2025
Shell

lorien /awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

crawler spider scraping crawling web-scraping captcha-recaptcha webscraping crawling-framework scraping-framework captcha-bypass scraping-tool crawling-tool scraping-python crawling-python

UpdatedDec 27, 2024
Makefile

autoscraper

alirezamika /autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

python crawler machine-learning scraper automation ai scraping artificial-intelligence web-scraping scrape webscraping webautomation

UpdatedJun 9, 2025
Python

Scrapling

D4Vinci /Scrapling

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

python crawler data automation ai scraping crawling web-scraper python3 web-scraping selectors xpath data-extraction stealth webscraping hacktoberfest crawling-python playwright web-scraping-python ai-scraping

UpdatedJul 10, 2025
Python

autoscrape-labs /pydoll

Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.

python webdriver asynchronous selenium chromium webscraping captcha-breaking cdp browser-automation bot-detection selenium-python puppeteer bypasscaptcha recaptcha-v3 anti-detection playwright turnstile-bypass

UpdatedJul 16, 2025
Python

niespodd /browser-fingerprinting

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?

bot crawler scraper automation recaptcha web spider detection chromium chromedriver fingerprinting browser-fingerprinting stealth webscraping bot-detection chromium-browser puppeteer

UpdatedJul 17, 2024
JavaScript

jaypyles /Scraperr

Self-hosted webscraper.

python docker kubernetes opensource helm scraping webscraper web-scraper self-hosted web-scraping web-scrapers webscraping playwright

UpdatedJul 13, 2025
TypeScript

daijro /camoufox

🦊 Anti-detect browser

firefox networking scraping fingerprint webscraping playwright antidetect antidetect-browser

UpdatedMar 15, 2025
C++

scrapoxy /scrapoxy

Scrapoxy is a super proxies manager that orchestrates all your proxies into one place, rather than spreading management across multiple scrapers. It manages IP rotation and fingerprinting, and smartly routes traffic to avoid bans.

proxies webscraping blacklisting antibot

UpdatedJun 6, 2025

anaskhan96 /soup

Web Scraper in Go, similar to BeautifulSoup

go golang webscraper web-scraper beautifulsoup webscraping html-node

UpdatedNov 2, 2023
Go

itsOwen /CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

scraper web-scraper openai webscraping gemini-api llm llm-scraper ai-scraping

UpdatedJun 16, 2025
Python

reworkd /tarsier

Vision utilities for web interaction agents 👀

python ocr selenium webscraping pypi-package playwright llms gpt4v

UpdatedNov 25, 2024
Jupyter Notebook

TheWebScrapingClub /webscraping-from-0-to-hero

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

python scrapy-spider scrapy webscraping scrapysplash playwright

UpdatedMay 27, 2024

jamesturk /scrapeghost

👻 Experimental library for scraping websites using OpenAI's GPT API.

gpt webscraping openai-api

UpdatedJun 17, 2025
Python

requests-cache

requests-cache /requests-cache

Persistent HTTP cache for python requests

redis http performance web mongodb sqlite cache dynamodb requests webscraping

UpdatedJul 12, 2025
Python

m8sec /CrossLinked

LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping

osint python3 enumeration webscraping pentest-scripts linkedin-scraper pentest-tool username-generator

UpdatedNov 26, 2024
Python

Improve this page

Add a description, image, and links to thewebscraping topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thewebscraping topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp