Copy of an email received on the Python Security Response team, 9 days ago. I consider that it's not worth it to have an embargo on this vulnerability, so I make it public.Hi there,I believe I've found a denial-of-service (DoS) bug inurllib.request.AbstractBasicAuthHandler. To start, I'm operating on somebackground information from this document: HTTP authentication<https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication>. The bugitself is a ReDoS <https://www.regular-expressions.info/redos.html> bugcausing catastrophic backtracking. To reproduce the issue we can use thefollowing code:from urllib.request import AbstractBasicAuthHandlerauth_handler = AbstractBasicAuthHandler()auth_handler.http_error_auth_reqed(    'www-authenticate',    'unused',    'unused',    {        'www-authenticate': 'Basic ' + ',' * 64 + ' ' + 'foo' + ' ' +'realm'    })The issue itself is in the following regular expression:rx = re.compile('(?:.*,)*[ \t]*([^ \t]+)[ \t]+'                'realm=(["\']?)([^"\']*)\\2', re.I)In particular, the (?:.*,)* portion. Since "." and "," overlap and thereare nested quantifiers we can cause catastrophic backtracking by repeatinga comma. Note that since AbstractBasicAuthHandler is vulnerable, then bothHTTPBasicAuthHandler and ProxyBasicAuthHandler are as well because theycall http_error_auth_reqed. Building from the HTTP authentication documentabove, this means a server can send a specially crafted header along withan HTTP 401 or HTTP 407 and cause a DoS on the client.I won't speculate on the severity of the issue too much - you will surelyunderstand the impact better than I will. Although, the fact that this isclient-side as opposed to server-side appears to reduce the severity,however the fact that it's a security-sensitive context (HTTPauthentication) may raise the severity.One possible fix would be changing the rx expression to the following:rx = re.compile('(?:[^,]*,)*[ \t]*([^ \t]+)[ \t]+'                'realm=(["\']?)([^"\']*)\\2', re.I)This removes the character overlap in the nested quantifier and thusnegates the catastrophic backtracking.Let me know if you have any questions or what the next steps are from here.Thanks for supporting Python security!-- Matt Schwager

I added this vulnerability to the following page to track fixes in all Python supported branches:https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html

CVE-2020-8492 has been assigned to this vulnerability:https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-8492

Isn't this a duplicate ofbpo-38826 ?

> Isn't this a duplicate ofbpo-38826 ?Oh right. I marked it as a duplicate of this issue.

bench_parser.py: Benchmark for AbstractBasicAuthHandler.http_error_auth_reqed().

Instead ofrepeat_10_3 = 'Basic ' + ', ' * (10 ** 3) + simplein the benchmark, tryrepeat_10_3 = 'Basic ' + ', ' * (10 ** 3) + 'A'

Ooooh, I see. I didn't measure the performance of the right header. I re-run a benchmark using the HTTP header (repeat=15):    header = 'Basic ' + ', ' * 15 + 'A'Now I see a major performance difference. Comparison between master ("ref") andPR 18284 ("fix"):Mean +- std dev: [ref] 88.9 ms +- 2.4 ms -> [fix] 17.5 us +- 0.7 us: 5083.23x faster (-100%)So the worst case is now way faster: more than 5000x faster!It's even possible to go up to repeat=10**6 characters, it still takes less than 1 seconds: 412 ms +- 19 ms.On the master branch, repeat=20 already takes around 3 seconds... The slowdown is exponential with repeat increase.

New changeset0b297d4ff1c0e4480ad33acae793fbaf4bf015b4 by Victor Stinner in branch 'master':bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284)https://github.com/python/cpython/commit/0b297d4ff1c0e4480ad33acae793fbaf4bf015b4

New changesetea9e240aa02372440be8024acb110371f69c9d41 by Miss Islington (bot) in branch '3.8':bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) (GH-19296)https://github.com/python/cpython/commit/ea9e240aa02372440be8024acb110371f69c9d41

New changesetb57a73694e26e8b2391731b5ee0b1be59437388e by Miss Islington (bot) in branch '3.7':bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) (GH-19297)https://github.com/python/cpython/commit/b57a73694e26e8b2391731b5ee0b1be59437388e

New changeset69cdeeb93e0830004a495ed854022425b93b3f3e by Victor Stinner in branch '3.6':bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) (GH-19304)https://github.com/python/cpython/commit/69cdeeb93e0830004a495ed854022425b93b3f3e

New changeset37fe316479e0b6906a74b0c0a5e495c55037fdfd by Victor Stinner in branch '3.5':bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) (#19305)https://github.com/python/cpython/commit/37fe316479e0b6906a74b0c0a5e495c55037fdfd

https://github.com/python/cpython/blob/9f93018b69d72cb48d3444554261ae3b0ea00c93/Lib/urllib/request.py#L989"headers" is a dict object? If so, the dict object does not seem to have no attribute "get_all".

> "headers" is a dict object? If so, the dict object does not seem to have no attribute "get_all".No, it's not a dict object.

At the beginning of the issue, there is the following reproduction code:from urllib.request import AbstractBasicAuthHandlerauth_handler = AbstractBasicAuthHandler()auth_handler.http_error_auth_reqed(    'www-authenticate',    'unused',    'unused',    {        'www-authenticate': 'Basic ' + ',' * 64 + ' ' + 'foo' + ' ' +'realm'    })Here's the headers:{        'www-authenticate': 'Basic ' + ',' * 64 + ' ' + 'foo' + ' ' +'realm' }I think this is a dict object, so the current problem is fixed and no longer compatible with the previous usage?

This issue was a security vulnerability. It's now closed, please don't comment closed issues. If you consider that there is a regression, please open a new issue.