Currently when a standard library http client (theurllib,urllib2,http, andhttplib modules) encounters anhttps:// URL it will wrapthe network HTTP traffic in a TLS stream, as is necessary to communicate withsuch a server. However, during the TLS handshake it will not actually checkthat the server has an X509 certificate is signed by a CA in any trust root,nor will it verify that the Common Name (or Subject Alternate Name) on thepresented certificate matches the requested host.
The failure to do these checks means that anyone with a privileged networkposition is able to trivially execute a man in the middle attack against aPython application using either of these HTTP clients, and change traffic atwill.
This PEP proposes to enable verification of X509 certificate signatures, aswell as hostname verification for Python’s HTTP clients by default, subject toopt-out on a per-call basis. This change would be applied to Python 2.7, Python3.4, and Python 3.5.
The “S” in “HTTPS” stands for secure. When Python’s users type “HTTPS” they areexpecting a secure connection, and Python should adhere to a reasonablestandard of care in delivering this. Currently we are failing at this, and indoing so, APIs which appear simple are misleading users.
When asked, many Python users state that they were not aware that Python failedto perform these validations, and are shocked.
The popularity ofrequests (which enables these checks by default)demonstrates that these checks are not overly burdensome in any way, and thefact that it is widely recommended as a major security improvement over thestandard library clients demonstrates that many expect a higher standard for“security by default” from their tools.
The failure of various applications to note Python’s negligence in this matteris a source ofregular CVE assignment[1][2][3][4][5][6][7][8][9][10][11].
Python would use the system provided certificate database on all platforms.Failure to locate such a database would be an error, and users would need toexplicitly specify a location to fix it.
This will be achieved by adding a newssl._create_default_https_contextfunction, which is the same asssl.create_default_context.
http.client can then replace its usage ofssl._create_stdlib_contextwith thessl._create_default_https_context.
Additionallyssl._create_stdlib_context is renamedssl._create_unverified_context (an alias is kept around for backwardscompatibility reasons).
This PEP proposes using the system-provided certificate database. Previousdiscussions have suggested bundling Mozilla’s certificate database and usingthat by default. This was decided against for several reasons:
OpenSSL also has a pair of environment variables,SSL_CERT_DIR andSSL_CERT_FILE which can be used to point Python at a different certificatedatabase.
This change will have the appearance of causing some HTTPS connections to“break”, because they will now raise an Exception during handshake.
This is misleading however, in fact these connections are presently failingsilently, an HTTPS URL indicates an expectation of confidentiality andauthentication. The fact that Python does not actually verify that the user’srequest has been made is a bug, further: “Errors should never pass silently.”
Nevertheless, users who have a need to access servers with self-signed orincorrect certificates would be able to do so by providing a context withcustom trust roots or which disables validation (documentation should stronglyrecommend the former where possible). Users will also be able to add necessarycertificates to system trust stores in order to trust them globally.
Twisted’s 14.0 release made this same change, and it has been met with almostno opposition.
For users who wish to opt out of certificate verification on a singleconnection, they can achieve this by providing thecontext argument tourllib.urlopen:
importssl# This restores the same behavior as before.context=ssl._create_unverified_context()urllib.urlopen("https://no-valid-cert",context=context)
It is also possible,though highly discouraged, to globally disableverification by monkeypatching thessl module in versions of Python thatimplement this PEP:
importssltry:_create_unverified_https_context=ssl._create_unverified_contextexceptAttributeError:# Legacy Python that doesn't verify HTTPS certificates by defaultpasselse:# Handle target environment that doesn't support HTTPS verificationssl._create_default_https_context=_create_unverified_https_context
This guidance is aimed primarily at system administrators that wish to adoptnewer versions of Python that implement this PEP in legacy environments thatdo not yet support certificate verification on HTTPS connections. Forexample, an administrator may opt out by adding the monkeypatch above tositecustomize.py in their Standard Operating Environment for Python.Applications and libraries SHOULD NOT be making this change process wide(except perhaps in response to a system administrator controlled configurationsetting).
Particularly security sensitive applications should always provide an explicitapplication defined SSL context rather than relying on the default behaviourof the underlying Python implementation.
This PEP only proposes requiring this level of validation for HTTP clients, notfor other protocols such as SMTP.
This is because while a high percentage of HTTPS servers have correctcertificates, as a result of the validation performed by browsers, for otherprotocols self-signed or otherwise incorrect certificates are far more common.Note that for SMTP at least, this appears to be changing and should be reviewedfor a potential similar PEP in the future:
This PEP describes changes that will occur on both the 3.4.x, 3.5 and 2.7.Xbranches. For 2.7.X this will require backporting thecontext(SSLContext) argument tohttplib, in addition to the features alreadybackported inPEP 466.
context argument tourlib.request.urlopen.This document has been placed into the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0476.rst
Last modified:2025-02-01 08:59:27 GMT