python-web-scraper

Star

Here are 62 public repositories matching this topic...

Language:All

Filter by language

All62 Python46 Jupyter Notebook4 HTML3 JavaScript2 CSS1 Java1 Swift1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

d60 /twikit

Sponsor

Star3.8k

python search bot client wrapper twitter-bot scraper twitter twitter-api scraping python3 twitter-client scrape tweepy x twitter-scraper python-web-scraper x-api twitter-internal-api

UpdatedJul 6, 2025
Python

omkarcloud /botasaurus

Sponsor

Star3.5k

The All in One Framework to Build Undefeatable Scrapers

anti-bot web-crawling bot-detection python-scraper anti-detect undetected scraping-framework undetectable python-web-scraper scraping-tool cloudflare-bypass scraping-python python-web-scraping anti-detection cloudflare-scrape bypass-cloudflare web-scraping-python undetected-chromedriver antidetect-browser anti-detect-browser

UpdatedDec 18, 2025
Python

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

web-scraper web-scraping newsletter reuters bloomberg futures web-scrapers scrapper financial-data news-websites data-scraping news-scraper futures-historical-data data-scraper sraping python-web-scraper financial-times options-data wall-street-journal wallstreetbets

UpdatedFeb 1, 2022
Python

oxylabs /scrape-google-python

Star793

In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.

python google scraping web-scraping scrape python-web-scraper scraping-api scraper-api python-scraping scrape-google

UpdatedSep 23, 2025

davidteather /everything-web-scraping

Sponsor

Star439

Learn everything web scraping with David Teather Codes on YouTube

python course everything reverse-engineering python3 web-scraping courses webscraping hacktoberfest youtube-series python-web-scraper project-based-learning web-scraping-tutorial project-based-learning-courses hacktoerfest web-scraping-python project-based-tutorials

UpdatedJul 31, 2023
HTML

jpjacobpadilla /Stealth-Requests

Star326

Undetected web-scraping & seamless HTML parsing in Python!

python data web-crawler http-client http-requests requests web-scraping xpath data-extraction html-parsing webscraping python-web-scraper python-scraping

UpdatedJul 14, 2025
Python

oxylabs /Python-Web-Scraping-Tutorial

Star300

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.

python crawler scraping web-scraping python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python amazon-scraper-python json-database-python

UpdatedSep 24, 2025
Python

oxylabs /aliexpress-scraper

Star230

Easily collect public AliExpress data like prices, product details, and ratings with AliExpress Scraper API

aliexpress aliexpress-crawler aliexpress-spider e-commerce-api email-crawler aliexpress-api python-web-scraper aliexpress-scraper scraper-api alibaba-api e-commerce-scraper github-python ali-scraper

UpdatedSep 25, 2025
Python

thewebscraping /tls-requests

Sponsor

Star117

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

anti-bot-page anti-bot python-web-crawler python-crawler python-spider web-crawler-python web-spider python-scraper tls-client python-web-scraper cloudflare-bypass scraping-python python-web-scraping crawling-python web-scraping-python anti-bot-detection cf-clearance cloudflare-scraper web-scraping-api python-tls-client

UpdatedNov 23, 2025
Python

oxylabs /web-scraping-google-sheets

Star83

Guide to Using Google Sheets for Basic Web Scraping

web-scraping web-crawler-python email-scraper google-trends-api python-web-scraper scraper-api web-scraping-python google-search-scraper google-sheets-web-scraping web-scraping-google-sheets

UpdatedSep 23, 2025

Conso1eCowb0y /Deepminer

Star51

Deep web crawler and search engine

github search-engine security crawler data-mining osint spider crawling tor hacking python3 onion tor-network webcrawler security-tools dark-web deepweb the-onion-router python-web-scraper deepminer

UpdatedAug 4, 2020
Python

oxylabs /ebay-scraper

Star38

eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

ebay url-scraper ebay-api python-web-scraper ebay-scraper ebay-search ebay-apis ebay-searches scrape-ebay

UpdatedAug 28, 2025
Java

J4NN0 /linkedin-web-scraper

Star38

LinkedIn Python Web Scraper to collect and store company data

scraper selenium webscraper scrapy-spider scrapy selenium-webdriver webscraping scraping-websites openpyxl scrapy-crawler scrapy-tutorial scrapy-demo selenium-python webscraper-website webscraping-search webscraper-api python-web-scraper python-excel

UpdatedDec 18, 2023
Python

stabldev /mangareader-api

Star34

A Python based web scraping api built with fastapi to get manga contents.

python anime scraping manga web-scraping mangareader python-web-scraper fastapi manga-api

UpdatedDec 16, 2025
Python

Decodo /Python-scraper-tutorial

Star33

A short introduction to scraping with Python with given steps and an example scraper script.

python learning crawler data-science data-mining scraping web-scraping beautifulsoup python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python json-database-python

UpdatedMay 14, 2025
Python

sallamy2580 /python-web-scrapping

Star30

scraping web-scraper newsletter reuters bloomberg scrapers futures web-scrapers financial-data news-websites data-scraping news-scraper futures-historical-data web-scrapping data-scraper python-web-scraper financial-times options-data wallstreetbets wall-street-jo

UpdatedJul 21, 2022
Python

JohnScooby /Google-Scraper

Star25

A Simple Python Based Google Search Results Scraper.

python scraper webscraping scrapy-crawler google-search dorker google-search-using-python python-web-scraper google-dorking dorking google-scraping web-scraping-project

UpdatedOct 31, 2022
Python

Decodo /Google-News-scraper

Star25

📰 Python Google News scraper for headlines, summaries & sources. Media monitoring, trend analysis with proxy rotation & automated CSV export.

python requests web-scraping data-extraction beautifulsoup google-scraper google-news google-news-scraper news-scraper trend-analysis news-analytics media-monitoring python-web-scraper proxy-rotation google-news-api scraping-google-trends headless-scraping decodo

UpdatedMay 13, 2025
Python

kowshika-n /Python-Scrapers

Star23

This repo includes a collection of Python scripts and tools built for enabling web scraping and data entry. Results are extracted and exported to CSV or Excel files as per requirements.

python automation python-library python-script webscraper freelance scripts-collection python-scripts webscraping freelancing freelancer standalone-snippets python-tools python-web-scraper python-scrapers