- Notifications
You must be signed in to change notification settings - Fork845
WiP: Fix looping Cloudflare challenge, Resolves #1036#1163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Conversation
garfield69 commentedApr 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
FWIW But after each solve there remains a chrome subtask that starts to spin up to 15% CPU and I have to manually kill them off. |
juanfrilla commentedApr 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Another thing that I've noticed is that in the user-agent headless replacement: self.execute_cdp_cmd("Network.setUserAgentOverride", {"userAgent":self.execute_script("return navigator.userAgent" ).replace("Headless","") }, ) I don't know why but If I hardcode the user-agent using the exact that my computer has like this: user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"options.add_argument(f"--user-agent={user_agent}") it bypasses cloudflare, but if i put this to make it automatically like you have it on line 533 from So an alternative could be to setup a driver only to get the user agent: defget_user_agent(driver):returndriver.execute_script("return navigator.userAgent;").replace("Headless","") And then pass the user-agent to the definitive driver PD: I only can tell you what I've discovered to see if we can go through the solution cuz I'm having troubles to get the project installed/set up 😅 |
m33ts4k0z commentedApr 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I didnt actually use this branch, it worked fine after I switched to it. Thanks |
ilike2burnthing commentedApr 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@garfield69 yea this seems to be an issue with Chrome v124. You can revert to v123 in the mean time if it's easier -#1161 Alternatively, build your own binaries, which will use Chromium v123: pip install -r requirements.txtpip install pyinstallercd srcpython build_package.py |
@m33ts4k0z were you doing this on Windows? |
m33ts4k0z commentedApr 21, 2024
Yes on a Windows 11 VM on Unraid but it did work in the end. I updated my first post here with the cause. |
Oh cool, did not know I could build on windows. |
ilike2burnthing commentedApr 22, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@juanfrilla sorry for the delay in replying, been busy and only got to a few quick ones on my phone. I'll have a look at the UA idea when I next get a chance, thanks. Assuming you're following the run from source instructions, what issue are you having?https://github.com/FlareSolverr/FlareSolverr#from-source-code |
juanfrilla commentedApr 22, 2024
@ilike2burnthing my main problem is that i cannot install Xvfb on MacOS |
Tried XQuartz? |
juanfrilla commentedApr 22, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
yessir now the project is set up, let's see what I can fix |
What exactly is left to do on this to get it merge? I tried to guess with the comments here and some different issues but I can't get the current status of this. It seems to be stale for quite some time, so what's needed? |
|
Well, I made my own implementation of this "new tab" idea and I was able to make it work with every website I could (ext.to, www3.yggtorrent.cool, dodi-repacks.site, hd-torrents.me/login.php, nhentai.net) on my Linux system using a VPN / socks5 proxy and also with my container image on my own remote Linux server, which was blocked by cloudflare too. Public image with my edits:21hsmw/flaresolverr:fixlooping |
That's working 95% of the time on Windows for me, even with a proxy, but failing 95% of the time on Docker. Usual error:
Seems it's related to |
When you say it fails on Docker, is it still on Windows or Linux? I got this error on Linux while doing my implementation, but have not been able to replicate it since. For the looping challenges, it seems to be a timing issue. Playing with the timer values can make it work in some cases, but it's not easy to know what works for everyone since it seems to take network latency into account. For example, if I use a proxy close to my location, it works 100% of the time with the sites I listed earlier, but if I use a proxy very far from me, it works 50% of the time. |
Linux. I'll play around with timings again (I did a bunch yesterday), see if I can get something that works both on my Docker and Windows. |
Strange then. I'm able to solve the challenges of all sites I try on my Debian and Fedora systems with different VPNs/Proxies with and without Docker involved. Here's an example with dodi-repacks.site using the docker image I shared previously: |
This comment was marked as off-topic.
This comment was marked as off-topic.
zenderzender commentedMay 3, 2024
Thanks for your workaround@21hsmw Working with @aevrard the solution you provide will kill the killswitch if you're using something like gluetun... |
LoicBrison commentedMay 3, 2024
Thanks@21hsmw ! |
vrozaksen commentedMay 3, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Worked for me on whatbox.ca services:flaresolverr:image:21hsmw/flaresolverr:fixloopingenvironment: -LOG_LEVEL=${LOG_LEVEL:-info} -LOG_HTML=${LOG_HTML:-false} -CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none} -TZ=UTC -PORT=25000 -HOST=127.0.0.1network_mode:hostpull_policy:alwaysrestart:unless-stopped |
juanfrilla commentedMay 3, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
replacing the image of the dockerfile for this: I tested as well on a centOS server with the previous image ( |
This comment was marked as duplicate.
This comment was marked as duplicate.
S-C-H commentedFeb 11, 2025
I believe@21hsmw you are aware of this one - although reading through the comments on this discussion, I did see some attempts were made to fix sessions. I can create a session but when I try to destroy it, it errors and then the task queue grows (doesn't complete).
|
@21hsmw looks like the Docker version is being detected, whereas running from source on Windows is working fine. Basically the challenge is found, checkbox is clicked, but challenge just reloads. I tried an apt-get update and upgrade, and restarting the container, but no luck. |
laur89 commentedFeb 21, 2025
Want to compare dockerized vs native performance; running locally however gives me: $ python -u src/flaresolverr.py Traceback (most recent call last): File"/tmp/FlareSolverr/src/flaresolverr.py", line 7,in<module> from bottle import run, response, Bottle, request, ServerAdapter File"/home/laur89/.pyenv/versions/flaresolverr/lib/python3.13/site-packages/bottle.py", line 38,in<module> import base64, cgi, email.utils, functools, hmac, itertools, mimetypes,\ os, re, subprocess, sys, tempfile, threading, time, warnings, hashlibModuleNotFoundError: No module named'cgi' Any pointers? |
Mylloon commentedFeb 21, 2025
either downgrade python to 3.11 or install legacy-cgi with pip, i guess |
nrg101 commentedFeb 21, 2025
"It was removed in Python 3.13 after being deprecated in Python 3.11. |
This comment was marked as spam.
This comment was marked as spam.
@tifo71 you have been warned plenty, and been around for long enough to see others warned as well. The issue has already been reported, your comment was unnecessary. Stop posting what is essentially just spam. If you do so again, you'll be blocked. No, do not reply to this comment. |
This comment was marked as spam.
This comment was marked as spam.
cedricdu78 commentedFeb 22, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Flaresolverr full work for me on VM ubuntu with Google chrome (no docker) |
tifo71 commentedFeb 22, 2025
thanks, finally i put alexfozor's image and it works mais 21hsmw will probably do an update so that it works with its version |
Jojont54 commentedFeb 24, 2025
Hello, on my side alexfozor's image doesn't works I guess it's due to the fact I have an intel processor on my unraid station meaning I need 386 version or something like that. |
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
tifo71 commentedFeb 25, 2025
Yes, there are currently 2 versions that work. so Alexfozor's image: alexfozor/flaresolverr:pr-1300-experimental and another called Byparr: ghcr.io/thephaseless/byparr:latest the current image of@21hsmw does not work. Waiting for an update |
My branch has been updated after the recent captcha looping issue.
Image: |
mokhunv-paloalto commentedMar 4, 2025
Hi, does your solution work with non-interactive cloudflare challenges? Even with your latest changes, I get "Challenge not detected!" when trying to access a page |
You'll need to give an example URL for anyone to be able to help you. |
This comment was marked as spam.
This comment was marked as spam.
mokhunv-paloalto commentedMar 6, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
FlareSolverr doesn't currently work with this type of integrated check, just whole page challenges. |
This comment was marked as spam.
This comment was marked as spam.
thebetauser commentedApr 7, 2025
Would it be possible to set the chrome version to download/run via ENV variables? This is more convenient for docker users who pull from the repo instead of rebuild images. @21hsmw Any chance that you can release docker images for arm64 arch? |
Thanks to@juanfrilla for#1036 (comment).
Unfortunately, currently this only works on Windows, and the looping challenges return if using proxies or VPNs.