urllib3 is fundraising for HTTP/2 support!Useful methods for working withhttp.client, completely decoupled fromcode specific tourllib3.
At the very core, just like its predecessors, urllib3 is built on top ofhttp.client – the lowest level HTTP library included in the Pythonstandard library.
To aid the limited functionality of thehttp.client module, urllib3provides various helper methods which are used with the higher level componentsbut can also be used independently.
Bases:object
Retry configuration.
Each retry attempt will create a new Retry object with updated values, sothey can be safely reused.
Retries can be defined as a default for a pool:
retries=Retry(connect=5,read=2,redirect=5)http=PoolManager(retries=retries)response=http.request("GET","https://example.com/")
Or per-request (which overrides the default for the pool):
response=http.request("GET","https://example.com/",retries=Retry(10))
Retries can be disabled by passingFalse:
response=http.request("GET","https://example.com/",retries=False)
Errors will be wrapped inMaxRetryError unlessretries are disabled, in which case the causing exception will be raised.
total (int) –
Total number of retries to allow. Takes precedence over other counts.
Set toNone to remove this constraint and fall back on othercounts.
Set to0 to fail on the first retry.
Set toFalse to disable and implyraise_on_redirect=False.
connect (int) –
How many connection-related errors to retry on.
These are errors raised before the request is sent to the remote server,which we assume has not triggered the server to process the request.
Set to0 to fail on the first retry of this type.
read (int) –
How many times to retry on read errors.
These errors are raised after the request was sent to the server, so therequest may have side-effects.
Set to0 to fail on the first retry of this type.
redirect (int) –
How many redirects to perform. Limit this to avoid infinite redirectloops.
A redirect is a HTTP response with a status code 301, 302, 303, 307 or308.
Set to0 to fail on the first retry of this type.
Set toFalse to disable and implyraise_on_redirect=False.
status (int) –
How many times to retry on bad status codes.
These are retries made on responses, where status code matchesstatus_forcelist.
Set to0 to fail on the first retry of this type.
other (int) –
How many times to retry on other errors.
Other errors are errors that are not connect, read, redirect or status errors.These errors might be raised after the request was sent to the server, so therequest might have side-effects.
Set to0 to fail on the first retry of this type.
Iftotal is not set, it’s a good idea to set this to 0 to accountfor unexpected edge cases and avoid infinite retry loops.
allowed_methods (Collection) –
Set of uppercased HTTP method verbs that we should retry on.
By default, we only retry on methods which are considered to beidempotent (multiple requests with the same parameters end with thesame state). SeeRetry.DEFAULT_ALLOWED_METHODS.
Set to aNone value to retry on any verb.
status_forcelist (Collection) –
A set of integer HTTP status codes that we should force a retry on.A retry is initiated if the request method is inallowed_methodsand the response status code is instatus_forcelist.
By default, this is disabled withNone.
backoff_factor (float) –
A backoff factor to apply between attempts after the second try(most errors are resolved immediately by a second try without adelay). urllib3 will sleep for:
{backofffactor}*(2**({numberofpreviousretries}))
seconds. Ifbackoff_jitter is non-zero, this sleep is extended by:
random.uniform(0,{backoffjitter})
seconds. For example, if the backoff_factor is 0.1, thenRetry.sleep() willsleep for [0.0s, 0.2s, 0.4s, 0.8s, …] between retries. No backoff will everbe longer thanbackoff_max.
By default, backoff is disabled (factor set to 0).
raise_on_redirect (bool) – Whether, if the number of redirects isexhausted, to raise a MaxRetryError, or to return a response with aresponse code in the 3xx range.
raise_on_status (bool) – Similar meaning toraise_on_redirect:whether we should raise an exception, or return a response,if status falls instatus_forcelist range and retries havebeen exhausted.
history (tuple) – The history of the request encountered duringeach call toincrement(). The list is in the orderthe requests occurred. Each list item is of classRequestHistory.
respect_retry_after_header (bool) – Whether to respect Retry-After header on status codes defined asRetry.RETRY_AFTER_STATUS_CODES or not.
remove_headers_on_redirect (Collection) – Sequence of headers to remove from the request when a responseindicating a redirect is returned before firing off the redirectedrequest.
backoff_max (float)
backoff_jitter (float)
Default methods to be used forallowed_methods
Default maximum backoff time.
Default headers to be used forremove_headers_on_redirect
Default status codes to be used forstatus_forcelist
Backwards-compatibility for the old retries format.
Get the value of Retry-After in seconds.
response (BaseHTTPResponse)
float | None
Return a new Retry object with incremented retry counters.
response (BaseHTTPResponse) – A response object, or None, if the server did notreturn a response.
error (Exception) – An error encountered during the request, orNone if the response was received successfully.
method (str |None)
url (str |None)
_pool (ConnectionPool |None)
_stacktrace (TracebackType |None)
A newRetry object.
Self
Is this method/status code retryable? (Based on allowlists and controlvariables such as the number of total retries to allow, whether torespect the Retry-After header, whether this header is present, andwhether the returned status code is on the list of status codes tobe retried upon on the presence of the aforementioned header)
Sleep between retry attempts.
This method will respect a server’sRetry-After response headerand sleep the duration of the time requested. If that is not present, itwill use an exponential backoff. By default, the backoff factor is 0 andthis method will return immediately.
response (BaseHTTPResponse |None)
None
Bases:_SSLContext
An SSLContext holds various SSL-related configuration options anddata, such as certificates and possibly a private key.
Bases:object
Timeout configuration.
Timeouts can be defined as a default for a pool:
importurllib3timeout=urllib3.util.Timeout(connect=2.0,read=7.0)http=urllib3.PoolManager(timeout=timeout)resp=http.request("GET","https://example.com/")print(resp.status)
Or per-request (which overrides the default for the pool):
response=http.request("GET","https://example.com/",timeout=Timeout(10))
Timeouts can be disabled by setting all the parameters toNone:
no_timeout=Timeout(connect=None,read=None)response=http.request("GET","https://example.com/",timeout=no_timeout)
This combines the connect and read timeouts into one; the read timeoutwill be set to the time leftover from the connect attempt. In theevent that both a connect timeout and a total are specified, or a readtimeout and a total are specified, the shorter timeout will be applied.
Defaults to None.
connect (int,float, orNone) – The maximum amount of time (in seconds) to wait for a connectionattempt to a server to succeed. Omitting the parameter will default theconnect timeout to the system default, probablythe global defaulttimeout in socket.py.None will set an infinite timeout for connection attempts.
The maximum amount of time (in seconds) to wait between consecutiveread operations for a response from the server. Omitting the parameterwill default the read timeout to the system default, probablytheglobal default timeout in socket.py.None will set an infinite timeout.
Note
Many factors can affect the total amount of time for urllib3 to returnan HTTP response.
For example, Python’s DNS resolver does not obey the timeout specifiedon the socket. Other factors that can affect total request time includehigh CPU load, high swap, the program running at a low priority level,or other behaviors.
In addition, the read and total timeouts only measure the time betweenread operations on the socket connecting the client and the server,not the total amount of time for the request to return a completeresponse. For most requests, the timeout is raised because the serverhas not sent the first byte in the specified time. This is not alwaysthe case; if a server streams one byte every fifteen seconds, a timeoutof 20 seconds will not trigger, even though the request will takeseveral minutes to complete.
A sentinel object representing the default timeout value
Create a copy of the timeout object
Timeout properties are stored per-pool but each request needs a freshTimeout object to ensure each one has its own start/stop configured.
a copy of the timeout object
Get the value to use when setting a connection timeout.
This will be a positive float or integer, the value None(never timeout), or the default system timeout.
Connect timeout.
int, float,Timeout.DEFAULT_TIMEOUT or None
Create a new Timeout from a legacy timeout value.
The timeout value used by httplib.py sets the same timeout on theconnect(), and recv() socket requests. This creates aTimeoutobject that sets the individual timeouts to thetimeout valuepassed to this function.
timeout (integer, float,urllib3.util.Timeout.DEFAULT_TIMEOUT, or None) – The legacy timeout value.
Timeout object
Gets the time elapsed since the call tostart_connect().
Elapsed time in seconds.
urllib3.exceptions.TimeoutStateError – if you attemptto get duration for a timer that hasn’t been started.
Get the value for the read timeout.
This assumes some time has elapsed in the connection timeout andcomputes the read timeout appropriately.
If self.total is set, the read timeout is dependent on the amount oftime taken by the connect timeout. If the connection time has not beenestablished, aTimeoutStateError will beraised.
Value to use for the read timeout.
urllib3.exceptions.TimeoutStateError – Ifstart_connect()has not yet been called on this object.
Start the timeout clock, used during a connect() attempt
urllib3.exceptions.TimeoutStateError – if you attemptto start a timer that has been started already.
Bases:Url
Data structure for representing an HTTP URL. Used as a return value forparse_url(). Both the scheme and host are normalized as they areboth case-insensitive according to RFC 3986.
Authority component as defined in RFC 3986 3.2.This includes userinfo (auth), host and port.
userinfo@host:port
Network location including host and port.
If you need the equivalent of urllib.parse’snetloc,use theauthority property instead.
Convert self into a url
This function should more or less round-trip withparse_url(). Thereturned url may not be exactly the same as the url inputted toparse_url(), but it should be equivalent by the RFC (e.g., urlswith a blank port will have : removed).
Example:
importurllib3U=urllib3.util.parse_url("https://google.com/mail/")print(U.url)# "https://google.com/mail/"print(urllib3.util.Url("https","username:password","host.com",80,"/path","query","fragment").url)# "https://username:password@host.com:80/path?query#fragment"
Checks if given fingerprint matches the supplied certificate.
Creates and configures anssl.SSLContext instance for use with urllib3.
ssl_version (int |None) –
The desired protocol version to use. This will default toPROTOCOL_SSLv23 which will negotiate the highest protocol that boththe server and your installation of OpenSSL support.
This parameter is deprecated instead use ‘ssl_minimum_version’.
ssl_minimum_version (int |None) – The minimum version of TLS to be used. Use the ‘ssl.TLSVersion’ enum for specifying the value.
ssl_maximum_version (int |None) – The maximum version of TLS to be used. Use the ‘ssl.TLSVersion’ enum for specifying the value.Not recommended to set to anything other than ‘ssl.TLSVersion.MAXIMUM_SUPPORTED’ which is thedefault value.
cert_reqs (int |None) – Whether to require the certificate verification. This defaults tossl.CERT_REQUIRED.
options (int |None) – Specific OpenSSL options. These default tossl.OP_NO_SSLv2,ssl.OP_NO_SSLv3,ssl.OP_NO_COMPRESSION, andssl.OP_NO_TICKET.
ciphers (str |None) – Which cipher suites to allow the server to select. Defaults to either system configuredciphers if OpenSSL 1.1.1+, otherwise uses a secure default set of ciphers.
verify_flags (int |None) – The flags for certificate verification operations. These default tossl.VERIFY_X509_PARTIAL_CHAIN andssl.VERIFY_X509_STRICT for Python 3.13+.
Constructed SSLContext object with specified options
Returns True if the connection is dropped and should be closed.:param conn:urllib3.connection.HTTPConnection object.
conn (BaseHTTPConnection)
Checks whether a given file-like object is closed.
Shortcuts for generating request headers.
keep_alive (bool |None) – IfTrue, adds ‘connection: keep-alive’ header.
accept_encoding (bool |list[str]|str |None) – Can be a boolean, list, or string.True translates to ‘gzip,deflate’. If the dependencies forBrotli (either thebrotli orbrotlicffi package) and/orZstandard (thebackports.zstd package for Python before 3.14)algorithms are installed, then their encodings areincluded in the string (‘br’ and ‘zstd’, respectively).List will get joined by comma.String will be used as provided.
user_agent (str |None) – String representing the user-agent you want, such as“python-urllib3/0.6”
basic_auth (str |None) – Colon-separated username:password string for ‘authorization: basic …’auth header.
proxy_basic_auth (str |None) – Colon-separated username:password string for ‘proxy-authorization: basic …’auth header.
disable_cache (bool |None) – IfTrue, adds ‘cache-control: no-cache’ header.
Example:
importurllib3print(urllib3.util.make_headers(keep_alive=True,user_agent="Batman/1.0"))# {'connection': 'keep-alive', 'user-agent': 'Batman/1.0'}print(urllib3.util.make_headers(accept_encoding=True))# {'accept-encoding': 'gzip,deflate'}
Given a url, return a parsedUrl namedtuple. Best-effort isperformed to parse incomplete urls. Fields not provided will be None.This parser is RFC 3986 and RFC 6874 compliant.
The parser logic and helper functions are based heavily onwork done in therfc3986 module.
Partly backwards-compatible withurllib.parse.
Example:
importurllib3print(urllib3.util.parse_url('http://google.com/mail/'))# Url(scheme='http', host='google.com', port=None, path='/mail/', ...)print(urllib3.util.parse_url('google.com:80'))# Url(scheme=None, host='google.com', port=80, path=None, ...)print(urllib3.util.parse_url('/foo?bar'))# Url(scheme=None, host=None, port=None, path='/foo', query='bar', ...)
Resolves the argument to a numeric constant, which can be passed tothe wrap_socket function/method from the ssl module.Defaults tossl.CERT_REQUIRED.If given a string it is assumed to be the name of the constant in thessl module or its abbreviation.(So you can specifyREQUIRED instead ofCERT_REQUIRED.If it’s neitherNone nor a string we assume it is already the numericconstant which can directly be passed to wrap_socket.
like resolve_cert_reqs
All arguments except for server_hostname, ssl_context, tls_in_tls, ca_cert_data andca_cert_dir have the same meaning as they do when usingssl.create_default_context(),ssl.SSLContext.load_cert_chain(),ssl.SSLContext.set_ciphers() andssl.SSLContext.wrap_socket().
server_hostname (str |None) – When SNI is supported, the expected hostname of the certificate
ssl_context (ssl.SSLContext |None) – A pre-madeSSLContext object. If none is provided, one willbe created usingcreate_urllib3_context().
ciphers (str |None) – A string of ciphers we wish the client to support.
ca_cert_dir (str |None) – A directory containing CA certificates in multiple separate files, assupported by OpenSSL’s -CApath flag or the capath argument toSSLContext.load_verify_locations().
key_password (str |None) – Optional password if the keyfile is encrypted.
ca_cert_data (None |str |bytes) – Optional string containing CA certificates in PEM format suitable forpassing as the cadata parameter to SSLContext.load_verify_locations()
tls_in_tls (bool) – Use SSLTransport to wrap the existing socket.
sock (socket.socket)
keyfile (str |None)
certfile (str |None)
cert_reqs (int |None)
ca_certs (str |None)
ssl_version (int |None)
ssl.SSLSocket | SSLTransportType
Waits for reading to be available on a given socket.Returns True if the socket is readable, or False if the timeout expired.
