python-web-scraper

Star

Here are 60 public repositories matching this topic...

Language:All

Filter by language

All60 Python44 Jupyter Notebook4 HTML3 JavaScript2 CSS1 Java1 Swift1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

d60 /twikit

Sponsor

Star2.6k

python search bot client wrapper twitter-bot scraper twitter twitter-api scraping python3 twitter-client scrape tweepy x twitter-scraper python-web-scraper x-api twitter-internal-api

UpdatedApr 22, 2025
Python

omkarcloud /botasaurus

Sponsor

Star1.9k

The All in One Framework to Build Undefeatable Scrapers

anti-bot web-crawling bot-detection python-scraper anti-detect undetected scraping-framework undetectable python-web-scraper scraping-tool cloudflare-bypass scraping-python python-web-scraping anti-detection cloudflare-scrape bypass-cloudflare web-scraping-python undetected-chromedriver antidetect-browser anti-detect-browser

UpdatedMay 22, 2025
Python

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

web-scraper web-scraping newsletter reuters bloomberg futures web-scrapers scrapper financial-data news-websites data-scraping news-scraper futures-historical-data data-scraper sraping python-web-scraper financial-times options-data wall-street-journal wallstreetbets

UpdatedFeb 1, 2022
Python

davidteather /everything-web-scraping

Sponsor

Star389

Learn everything web scraping with David Teather Codes on YouTube

python course everything reverse-engineering python3 web-scraping courses webscraping hacktoberfest youtube-series python-web-scraper project-based-learning web-scraping-tutorial project-based-learning-courses hacktoerfest web-scraping-python project-based-tutorials

UpdatedJul 31, 2023
HTML

oxylabs /Python-Web-Scraping-Tutorial

Star282

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.

python crawler scraping web-scraping python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python amazon-scraper-python json-database-python

UpdatedApr 9, 2025
Python

jpjacobpadilla /Stealth-Requests

Star246

Undetected Web-Scraping & Seamless HTML Parsing in Python!

python data web-crawler http-client http-requests requests web-scraping xpath data-extraction html-parsing webscraping python-web-scraper python-scraping

UpdatedFeb 3, 2025
Python

oxylabs /aliexpress-scraper

Star231

Easily collect public AliExpress data like prices, product details, and ratings with AliExpress Scraper API

aliexpress aliexpress-crawler aliexpress-spider e-commerce-api email-crawler aliexpress-api python-web-scraper aliexpress-scraper scraper-api alibaba-api e-commerce-scraper github-python ali-scraper

UpdatedApr 9, 2025
Python

oxylabs /scrape-google-python

Star118

In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.

python google scraping web-scraping scrape python-web-scraper scraping-api scraper-api python-scraping scrape-google

UpdatedApr 9, 2025

oxylabs /web-scraping-google-sheets

Star81

Guide to Using Google Sheets for Basic Web Scraping

web-scraping google-news-scraper web-crawler-python email-scraper google-trends-api python-web-scraper google-news-api web-scraping-python google-search-scraper google-sheets-web-scraping web-scraping-google-sheets

UpdatedFeb 11, 2025

thewebscraping /tls-requests

Star55

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

anti-bot-page anti-bot python-web-crawler python-crawler python-spider web-crawler-python web-spider python-scraper tls-client python-web-scraper cloudflare-bypass scraping-python python-web-scraping crawling-python web-scraping-python anti-bot-detection cf-clearance cloudflare-scraper web-scraping-api python-tls-client

UpdatedMay 18, 2025
Python

Conso1eCowb0y /Deepminer

Star52

Deep web crawler and search engine

github search-engine security crawler data-mining osint spider crawling tor hacking python3 onion tor-network webcrawler security-tools dark-web deepweb the-onion-router python-web-scraper deepminer

UpdatedAug 4, 2020
Python

AbderrahimAl /Facebook-Scraper

Star42

Collect data from Facebook Posts based on search queries 🦋

python selenium bs4 facebook-scraper python-web-scraper

UpdatedApr 7, 2023
Python

J4NN0 /linkedin-web-scraper

Star39

Python Web Scraper for LinkedIn to collect and store company data (e.g. name, description, industry, etc.) into .xls file

scraper selenium webscraper scrapy-spider scrapy selenium-webdriver webscraping scraping-websites openpyxl scrapy-crawler scrapy-tutorial scrapy-demo selenium-python webscraper-website webscraping-search webscraper-api python-web-scraper python-excel

UpdatedDec 18, 2023
Python

oxylabs /ebay-scraper

Star33

eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

ebay url-scraper ebay-api python-web-scraper ebay-scraper ebay-search ebay-apis ebay-searches scrape-ebay

UpdatedApr 9, 2025
Java

Decodo /Python-scraper-tutorial

Star30

A short introduction to scraping with Python with given steps and an example scraper script.

python learning crawler data-science data-mining scraping web-scraping beautifulsoup python-web-crawler webscraping web-crawler-python python-web-scraper python-projects web-scraping-python github-python web-scraping-api scraper-python json-database-python

UpdatedMay 14, 2025
Python

sallamy2580 /python-web-scrapping

Star28

scraping web-scraper newsletter reuters bloomberg scrapers futures web-scrapers financial-data news-websites data-scraping news-scraper futures-historical-data web-scrapping data-scraper python-web-scraper financial-times options-data wallstreetbets wall-street-jo

UpdatedJul 21, 2022
Python

moonlitgrace /mangareader-api

Star26

A Python based web scraping api built with fastapi that provides easy access to manga contents

python anime scraping manga web-scraping mangareader python-web-scraper fastapi manga-api

UpdatedMay 19, 2025
Python

JohnScooby /Google-Scraper

Star25

A Simple Python Based Google Search Results Scraper.

python scraper webscraping scrapy-crawler google-search dorker google-search-using-python python-web-scraper google-dorking dorking google-scraping web-scraping-project

UpdatedOct 31, 2022
Python

kowshika-n /Python-Scrapers

Star23

This repo includes a collection of Python scripts and tools built for enabling web scraping and data entry. Results are extracted and exported to CSV or Excel files as per requirements.

python automation python-library python-script webscraper freelance scripts-collection python-scripts webscraping freelancing freelancer standalone-snippets python-tools python-web-scraper python-scrapers