- Notifications
You must be signed in to change notification settings - Fork2
user friendly wrapper for pycurl
License
Notifysolutions/request_curl
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
A user-friendly wrapper for pycurl that simplifies HTTP requests.
Use the package managerpipto installrequest_curl.
NOTE: You need Python and libcurl installed on your system to use or build pycurl. Some RPM distributions of curl/libcurl do not include everything necessary to build pycurl, in which case you need to install the developer specific RPM which is usually called curl-dev.
pip install request_curlA request_curl session manages cookies, connection pooling, and configurations.
Basic Usage:
importrequest_curls=request_curl.Session()s.get('https://httpbin.org/get')# returns <Response [200]>s.request('GET','https://httpbin.org/get')# returns <Response [200]>
Using a Context Manager
importrequest_curlwithrequest_curl.Session()assession:session.get('https://httpbin.org/get')# returns <Response [200]>
The response object is similar to that of therequests library.
importrequest_curls=request_curl.Session()r=s.get("https://httpbin.org/get")print(r)# prints response objectprint(r.status_code)# prints status codeprint(r.content)# prints response content in bytesprint(r.text)# prints response content as textprint(r.json)# prints response content as JSONprint(r.url)# prints response URLprint(r.headers)# prints response headers
Format the proxy as a string.
importrequest_curls=request_curl.Session()# supports authentication: r = s.get("https://httpbin.org/get", proxies="ip:port:user:password")r=s.get("https://httpbin.org/get",proxies="ip:port")
HTTP2 is disabled by default.
importrequest_curls=request_curl.Session(http2=True)r=s.get("https://httpbin.org/get")
You can specify custom cipher suites as an array.
importrequest_curlcipher_suite= ["AES128-SHA256","AES256-SHA256","AES128-GCM-SHA256","AES256-GCM-SHA384"]s=request_curl.Session(cipher_suite=cipher_suite)r=s.get("https://httpbin.org/get")
Set debug to True to print raw input and output headers.
importrequest_curls=request_curl.Session()r=s.get("https://httpbin.org/get",debug=True)
Specify custom headers as a dictionary.
importrequest_curls=request_curl.Session()headers= {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"}r=s.get("https://httpbin.org/get",headers=headers)
importrequest_curls=request_curl.Session()# sending form dataform_data= {"key":"value"}response=s.post("https://httpbin.org/post",data=form_data)# sending json datajson_data= {"key":"value"}response=s.post("https://httpbin.org/post",json=json_data)
To use request_curl withcurl-impersonate,opt for ourcustom Docker image by either pulling or building it.The image comes with request_curl and curl-impersonate pre-installed.Check below for a demonstration on impersonating firefox98 tls-fingerprint and request_curl with our custom Docker Image.
Note: This feature is still considered experimental. Only tested with firefox fingerprint
To pull the Docker image:
docker pull h3adex/request-curl-impersonate:latestdocker run --rm -it h3adex/request-curl-impersonate
Example Python code for a target website:
importrequest_curlfromrequest_curlimportFIREFOX98_CIPHER_SUITE,FIREFOX98_HEADERS# impersonates ff98session=request_curl.Session(http2=True,cipher_suite=FIREFOX98_CIPHER_SUITE,headers=FIREFOX98_HEADERS)response=session.get("https://tls.browserleaks.com/json")# <Response [200]># "ja3_hash":"25e9b0dd5b8e9330b206eae87e885e19"# same result as:# docker run --rm lwthiker/curl-impersonate:0.5-ff curl_ff98 https://tls.browserleaks.com/json
We welcome contributions through pull requests.Before making major changes, please open an issue to discuss your intended changes.Also, ensure to update relevant tests.
Ennis BlankEnnis.Blank@fau.de, Mauritz UphoffMauritz.Uphoff@hs-osnabrueck.de
About
user friendly wrapper for pycurl