PEP 476 updated Python’s default handling of HTTPS certificates in clientmodules to align with certificate handling in web browsers, by validatingthat the certificates received belonged to the server the client was attemptingto contact. The Python 2.7 long term maintenance series was judged to be inscope for this change, with the new behaviour introduced in the Python 2.7.9maintenance release.
This has created a non-trivial barrier to adoption for affected Python 2.7maintenance releases, so this PEP proposes additional Python 2.7 specificfeatures that allow system administrators and other users to more easilydecouple the decision to verify server certificates in HTTPS client modulesfrom the decision to update to newer Python 2.7 maintenance releases.
PEP 476 changed Python’s default behaviour to align with expectationsestablished by web browsers in regards to the semantics of HTTPS URLs:starting with Python 2.7.9 and 3.4.3, HTTPS clients in the standard libraryvalidate server certificates by default.
However, it is also the case that this changedoes cause problems forinfrastructure administrators operating private intranets that rely onself-signed certificates, or otherwise encounter problems with the new defaultcertificate verification settings.
To manage these kinds of situations, web browsers provide users with “clickthrough” warnings that allow the user to add the server’s certificate to thebrowser’s certificate store. Network client tools likecurl andwgetoffer options to switch off certificate checking entirely (by way ofcurl--insecure andwget--no-check-certificate, respectively).
At a different layer of the technology stack, Linux security modules likeSELinux andAppArmor, while enabled by default by distribution vendors,offer relatively straightforward mechanisms for turning them off.
At the moment, no such convenient mechanisms exist to disable Python’sdefault certificate checking for a whole process.
PEP 476 did attempt to address this question, by covering how to revert to theold settings process wide by monkeypatching thessl module to restore theold behaviour. Unfortunately, thesitecustomize.py based technique proposedto allow system administrators to disable the feature by default in theirStandard Operating Environment definition has been determined to beinsufficient in at least some cases. The specific case that led to theinitial creation of this PEP is the one where a Linux distributor aims toprovide their users with asmoother migration paththan the standard one provided by consuming upstream CPython 2.7 releasesdirectly, but other potential challenges have also been pointed out withupdating embedded Python runtimes and other user level installations of Python.
Rather than allowing a plethora of mutually incompatible migration techniquesto bloom, this PEP proposes an additional feature to be added to Python 2.7.12to make it easier to revert a process to the past behaviour of skippingcertificate validation in HTTPS client modules. It also provides additionalrecommendations to redistributors backporting these features to versions ofPython prior to Python 2.7.9.
In the absence of clear upstream guidance and recommendations, commercialredistributors will still make their own design decisions in the interests oftheir customers. The main approaches available are:
These changes are being proposed purely as tools for helping to manage thetransition to the new default certificate handling behaviour in the contextof Python 2.7. They are not being proposed as new features for Python 3, asit is expected that the vast majority of client applications affected by thisproblem without the ability to update the application itself will be Python 2applications.
It would likely be desirable for a future version of Python 3 to allow thedefault certificate handling for secure protocols to be configurable on aper-protocol basis, but that question is beyond the scope of this PEP.
As the proposals in this PEP aim to facilitate backports to earlier Pythonversions, the Python version number cannot be used as a reliable means fordetecting them. Instead, they are designed to allow the presenceor absence of the feature to be determined using the following technique:
python-c"import ssl; ssl.<_relevant_attribute>"
This will fail withAttributeError (and hence a non-zero return code) ifthe relevant capability is not available.
The feature detection attributes defined by this PEP are:
ssl._https_verify_certificates: runtime configuration APIssl._https_verify_envvar: environment based configurationssl._cert_verification_config: file based configuration (PEP 476 opt-in)The marker attributes are prefixed with an underscore to indicate theimplementation dependent and security sensitive nature of these capabilities.
This change is proposed for inclusion in CPython 2.7.12 and later CPython 2.7.xreleases. It consists of a newssl._https_verify_certificates() to specifythe default handling of HTTPS certificates in standard library client libraries.
It is not proposed to forward port this change to Python 3, so Python 3applications that need to support skipping certificate verification will stillneed to define their own suitable security context.
The marker attribute on thessl module related to this feature is thessl._https_verify_certificates function itself.
Thessl._https_verify_certificates function will work as follows:
def_https_verify_certificates(enable=True):"""Verify server HTTPS certificates by default?"""global_create_default_https_contextifenable:_create_default_https_context=create_default_contextelse:_create_default_https_context=_create_unverified_context
If called without arguments, or withenable set to a true value, thenstandard library client modules will subsequently verify HTTPS certificates by default, otherwise they will skip verification.
If called withenable set to a false value, then standard library clientmodules will subsequently skip verifying HTTPS certificates by default.
The inclusion of this feature will allow security sensitive applications toinclude the following forward-compatible snippet in their code:
ifhasattr(ssl,"_https_verify_certificates"):ssl._https_verify_certificates()
Some developers may also choose to opt out of certificate checking usingssl._https_verify_certificates(enable=False). This doesn’t introduce anymajor new security concerns, as monkeypatching the affected internal APIs wasalready possible.
This change is proposed for inclusion in CPython 2.7.12 and later CPython 2.7.xreleases. It consists of a newPYTHONHTTPSVERIFY environment variable thatcan be set to'0' to disable the default verification without modifying theapplication source code (which may not even be available in cases ofbytecode-only application distribution)
It is not proposed to forward port this change to Python 3, so Python 3applications that need to support skipping certificate verification will stillneed to define their own suitable security context.
The marker attribute on thessl module related to this feature is:
ssl._https_verify_envvar attribute, giving the name of environmentvariable affecting the default behaviourThis not only makes it straightforward to detect the presence (or absence) ofthe capability, it also makes it possible to programmatically determine therelevant environment variable name.
Rather than always defaulting to the use ofssl.create_default_context,thessl module will be modified to:
PYTHONHTTPSVERIFY environment variable when the module is firstimported into a Python processssl._create_default_https_context function to be an alias forssl._create_unverified_context if this environment variable is presentand set to'0'ssl._create_default_https_context function to be analias forssl.create_default_context as usual_https_verify_envvar='PYTHONHTTPSVERIFY'def_get_https_context_factory():ifnotsys.flags.ignore_environment:config_setting=os.environ.get(_https_verify_envvar)ifconfig_setting=='0':return_create_unverified_contextreturncreate_default_context_create_default_https_context=_get_https_context_factory()
Relative to the behaviour in Python 3.4.3+ and Python 2.7.9->2.7.11, thisapproach does introduce a new downgrade attack against the default securitysettings that potentially allows a sufficiently determined attacker to revertPython to the default behaviour used in CPython 2.7.8 and earlier releases.
This slight increase in the available attack surface is a key reason why:
However, it’s also worth keeping in mind that carrying out such an attackrequires the ability to modify the execution environment of a Python processprior to the import of thessl module. In combination with the abilityto write to any part of the filesystem (such as/tmp), any attacker withsuch access would already be able to modify the behaviour of the underlyingOpenSSL implementation, the dynamic library loader, and other potentiallysecurity sensitive components.
The default setting is read directly from the process environment, and henceworks the same way regardless of whether or not the interpreter is being runinside an activated Python virtual environment.
A patch for Python 2.7 implementing the above two features is attached totherelevant tracker issue.
If this PEP is accepted, then commercial Python redistributors may choose tobackport the per-process configuration mechanisms defined in this PEP to baseversions older than Python 2.7.9,without also backportingPEP 476’s changeto the default behaviour of the overall Python installation.
Such a backport would differ from the mechanism proposed in this PEP solely inthe default behaviour whenPYTHONHTTPSVERIFY was not set at all: it wouldcontinue to default to skipping certificate validation.
In this case, if thePYTHONHTTPSVERIFY environment variable is defined, andset to anythingother than'0', then HTTPS certificate verificationshould be enabled.
There’s no specific attribute indicating that this situation applies. Rather,it is indicated by thessl._https_verify_certificates andssl._https_verify_envvar attributes being present in a Python version thatis nominally older than Python 2.7.12.
Implementing this backport involves backporting the changes inPEP 466, 476 andthis PEP, with the following change to the handling of thePYTHONHTTPSVERIFY environment variable in thessl module:
PYTHONHTTPSVERIFY environment variable when the module is firstimported into a Python processssl._create_default_https_context function to be an alias forssl.create_default_context if this environment variable is presentand set to any value other than'0'ssl._create_default_https_context function to be analias forssl._create_unverified_context_https_verify_envvar='PYTHONHTTPSVERIFY'def_get_https_context_factory():ifnotsys.flags.ignore_environment:config_setting=os.environ.get(_https_verify_envvar)ifconfig_setting!='0':returncreate_default_contextreturn_create_unverified_context_create_default_https_context=_get_https_context_factory()def_disable_https_default_verification():"""Skip verification of HTTPS certificates by default"""global_create_default_https_context_create_default_https_context=_create_unverified_context
This change would be a strict security upgrade for any Python version thatcurrently defaults to skipping certificate validation in standard libraryHTTPS clients. The technical trade-offs to be taken into account relate largelyto the magnitude of thePEP 466 backport also required rather than to anythingsecurity related.
The default setting is read directly from the process environment, and henceworks the same way regardless of whether or not the interpreter is being runinside an activated Python virtual environment.
The backporting approach described above leaves the default HTTPS certificateverification behaviour of a Python 2.7 installation unmodified: verifyingcertificates still needs to be opted into on a per-connection or per-processbasis.
To allow the default behaviour of the entire installation to be modifiedwithout breaking backwards compatibility, Red Hat designed a configurationmechanism for the system Python 2.7 installation in Red Hat Enterprise Linux7.2+ that provides:
As it only affects backports to earlier releases of Python 2.7, this change isnot proposed for inclusion in upstream CPython, but rather is offered asa recommendation to other redistributors that choose to offer a similar featureto their users.
This PEP doesn’t take a position on whether or not this particular change is agood idea - rather, it suggests thatif a redistributor chooses to go downthe path of making the default behaviour configurable in a version of Pythonolder than Python 2.7.9, then maintaining a consistent approach acrossredistributors would be beneficial for users.
However, this approach SHOULD NOT be used for any Python installation thatadvertises itself as providing Python 2.7.9 or later, as most Python userswill have the reasonable expectation that all such environments will verifyHTTPS certificates by default.
The marker attribute on thessl module related to this feature is:
_cert_verification_config='<path to configuration file>'
This not only makes it straightforward to detect the presence (or absence) ofthe capability, it also makes it possible to programmatically determine therelevant configuration file name.
The recommended approach to backporting thePEP 476 modifications to an earlierpoint release is to implement the following changes relative to the defaultPEP 476 behaviour implemented in Python 2.7.9+:
ssl module to read a system wide configuration file when themodule is first imported into a Python processssl._create_default_https_context function to be an alias foreitherssl.create_default_context orssl._create_unverified_contextbased on the given configuration setting.As the PEP authors are not aware of any vendors providing long-term supportreleases targeting Windows, Mac OS X or *BSD systems, this approach iscurrently only specifically defined for Linux system Python installations.
The recommended configuration file name on Linux systems is/etc/python/cert-verification.cfg.
The.cfg filename extension is recommended for consistency with thepyvenv.cfg used by thevenv module in Python 3’s standard library.
The configuration file should use a ConfigParser ini-style format with asingle section named[https] containing one required settingverify.
The suggested section name is taken from the “https” URL schema passed toaffected client APIs.
Permitted values forverify are:
enable: ensure HTTPS certificate verification is enabled by defaultdisable: ensure HTTPS certificate verification is disabled by defaultplatform_default: delegate the decision to the redistributor providingthis particular Python versionIf the[https] section or theverify setting are missing, or if theverify setting is set to an unknown value, it should be treated as if theconfiguration file is not present.
_cert_verification_config='/etc/python/cert-verification.cfg'def_get_https_context_factory():# Check for a system-wide override of the default behaviourcontext_factories={'enable':create_default_context,'disable':_create_unverified_context,'platform_default':_create_unverified_context,# For now :)}importConfigParserconfig=ConfigParser.RawConfigParser()config.read(_cert_verification_config)try:verify_mode=config.get('https','verify')except(ConfigParser.NoSectionError,ConfigParser.NoOptionError):verify_mode='platform_default'default_factory=context_factories.get('platform_default')returncontext_factories.get(verify_mode,default_factory)_create_default_https_context=_get_https_context_factory()
The specific recommendations for this backporting case are designed to work forprivileged, security sensitive processes, even those being run in the followinglocked down configuration:
sys.path[0] based privilege escalation attacks)-E switch (preventingPYTHON* environment variable basedprivilege escalation attacks)-s switch (preventing user site directory based privilegeescalation attacks)-S switch (preventingsitecustomize based privilegeescalation attacks)The intent is that theonly reason HTTPS verification should be gettingturned off installation wide when using this approach is because:
Using an administrator controlled configuration file rather than an environmentvariable has the essential feature of providing a smoother migration path, evenfor applications being run with the-E switch.
This setting is scoped by the interpreter installation and affects all Pythonprocesses using that interpreter, regardless of whether or not the interpreteris being run inside an activated Python virtual environment.
This recommendation is based on the backporting approach adopted for Red HatEnterprise Linux 7.2, as published in the original July 2015 draft of this PEPand described in detail inthis KnowledgeBase article. Red Hat’s patches implementingthis backport for Python 2.7.5 can be found in theCentOS git repository.
If a redistributor chooses to backport the environment variable basedconfiguration setting from this PEP to a modified Python version that alsoimplements the configuration file basedPEP 476 backport, then the environmentvariable should take precedence over the system-wide configuration setting.This allows the setting to be changed for a given user or application,regardless of the installation-wide default behaviour.
_https_verify_envvar='PYTHONHTTPSVERIFY'_cert_verification_config='/etc/python/cert-verification.cfg'def_get_https_context_factory():# Check for an environmental override of the default behaviourifnotsys.flags.ignore_environment:config_setting=os.environ.get(_https_verify_envvar)ifconfig_settingisnotNone:ifconfig_setting=='0':return_create_unverified_contextreturncreate_default_context# Check for a system-wide override of the default behaviourcontext_factories={'enable':create_default_context,'disable':_create_unverified_context,'platform_default':_create_unverified_context,# For now :)}importConfigParserconfig=ConfigParser.RawConfigParser()config.read(_cert_verification_config)try:verify_mode=config.get('https','verify')except(ConfigParser.NoSectionError,ConfigParser.NoOptionError):verify_mode='platform_default'default_factory=context_factories.get('platform_default')returncontext_factories.get(verify_mode,default_factory)_create_default_https_context=_get_https_context_factory()
This document has been placed into the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0493.rst
Last modified:2025-02-01 08:59:27 GMT