- Notifications
You must be signed in to change notification settings - Fork244
A cloudflare verification bypass script for webscraping
License
sarperavci/CloudflareBypassForScraping
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
We love scraping, don't we? But sometimes, we face Cloudflare protection. This script is designed to bypass the Cloudflare protection on websites, allowing you to interact with them programmatically.
If you're looking for a long-term and stable solution to bypass Cloudflare, CAPTCHA challenges, and sophisticated bot detection systems, we highly recommendScrapeless Scraping Browser.
Scrapeless Scraping Browser offers low-level dynamic browser environment spoofing and automated CAPTCHA bypassing, significantly improving the stability, success rate, and anti-blocking capability of your project. It is especially well-suited for mid-to-large-scale scraping systems or commercial automation applications.
Key Advantages of Scrapeless Scraping Browser:
- Built-in CAPTCHA solving: Automatically bypasses Cloudflare Turnstile, reCAPTCHA, AWS WAF, DataDome, and other challenge systems.
- Undetectable browser environment: Not based on the traditional WebDriver — avoids automation detection.
- Massive concurrency support: Run 50–10,000+ browser instances simultaneously with no server constraints.
- Real-time debugging: Live View and session recording for efficient troubleshooting.
- Native integration: Compatible with Puppeteer, Playwright, Python, and Node.js — easy to integrate into your current workflows.
- 70M+ residential IPs: Global proxy network with automatic rotation and smart geolocation routing.
Scrapeless is an all-in-one, highly scalable data scraping platform built for enterprises and developers. In addition to the Scraping Browser, Scrapeless also offers aScraping API,Deep Serp API, andProxy services.👉 Learn more:Scrapeless Scraping Browser |Documentation
If you use Selenium, you may have noticed that it is not possible to bypass Cloudflare protection with it. Even you click the "I'm not a robot" button, you will still be stuck in the "Checking your browser before accessing" page.This is because Cloudflare protection is able to detect the automation tools and block them, which puts the webdriver infinitely in the "Checking your browser before accessing" page.
As you realize, the script uses the DrissionPage, which is a controller for the browser itself. This way, the browser is not detected as a webdriver and the Cloudflare protection is bypassed.
You can install the required packages by running the following command:
pip install -r requirements.txt
Create a new instance of theCloudflareBypass
class and call thebypass
method when you need to bypass the Cloudflare protection.
fromCloudflareBypasserimportCloudflareBypasserfromDrissionPageimportChromiumPagedriver=ChromiumPage()driver.get('https://nopecha.com/demo/cloudflare')cf_bypasser=CloudflareBypasser(driver)cf_bypasser.bypass()
You can run the test script to see how it works:
python test.py
Recently,@frederik-uni has introduced a new feature called "Server Mode". This feature allows you to bypass the Cloudflare protection remotely, either you can get the cookies or the HTML content of the website.
You can install the required packages by running the following command:
pip install -r server_requirements.txt
Start the server by running the following command:
python server.py
Two endpoints are available:
/cookies?url=<URL>&retries=<>&proxy=<>
: This endpoint returns the cookies of the website (including the Cloudflare cookies)./html?url=<URL>&retries=<>&proxy=<>
: This endpoint returns the HTML content of the website.
Send a GET request to the desired endpoint with the URL of the website you want to bypass the Cloudflare protection.
sarp@IdeaPad:~/$ curl http://localhost:8000/cookies?url=https://nopecha.com/demo/cloudflare{"cookies":{"cf_clearance":"SJHuYhHrTZpXDUe8iMuzEUpJxocmOW8ougQVS0.aK5g-1723665177-1.0.1.1-5_NOoP19LQZw4TQ4BLwJmtrXBoX8JbKF5ZqsAOxRNOnW2rmDUwv4hQ7BztnsOfB9DQ06xR5hR_hsg3n8xteUCw"},"user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"}
You can also run the server in a Docker container. Thanks to@gandrunx for Dockerizing the server.
First, build the Docker image:
docker build -t cloudflare-bypass.
Then, run the Docker container:
docker run -p 8000:8000 cloudflare-bypass
Alternatively, you can skipdocker build
step, and run the container using pre-build image:
docker run -p 8000:8000 ghcr.io/sarperavci/cloudflarebypassforscraping:latest
Here are some example projects that utilize the CloudflareBypasser Server:
- Calibre Web Automated Book Downloader - A tool to download books from calibre web.
- Kick Unofficial API - A tool to interact with the Kick.com, download videos, send messages, etc.
About
A cloudflare verification bypass script for webscraping
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
Packages0
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Contributors12
Uh oh!
There was an error while loading.Please reload this page.