Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

bpo-45001: Make email date parsing more robust against malformed input#27946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@wbolster
Copy link
Contributor

@wbolsterwbolster commentedAug 25, 2021
edited by bedevere-bot
Loading

Seehttps://bugs.python.org/issue45001

Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.

The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.

In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.

Here's a minimal example:

$ pythonPython 3.9.6 (default, Jun 30 2021, 10:22:16)[GCC 11.1.0] on linuxType "help", "copyright", "credits" or "license" for more information.>>> import email.utils>>> email.utils.parsedate('foo')>>> email.utils.parsedate(' ')Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate    t = parsedate_tz(data)  File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz    res = _parsedate_tz(data)  File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz    if data[0].endswith(',') or data[0].lower() in _daynames:IndexError: list index out of range

The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.

https://bugs.python.org/issue45001

gertvdijk reacted with heart emoji
Seehttps://bugs.python.org/issue45001Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.
@wbolsterwbolsterforce-pushed thebpo45001-date-email-parsing-robustness branch from0edbd8d tod75760cCompareAugust 25, 2021 14:34
@wbolsterwbolster changed the titlebpo45001: Make email date parsing more robust against malformed inputbpo-45001: Make email date parsing more robust against malformed inputAug 25, 2021
@ambv
Copy link
Contributor

Also added backport to 3.8 since passing invalid headers via e-mail can lead to DoS here.

gertvdijk and wbolster reacted with thumbs up emoji

@wbolster
Copy link
ContributorAuthor

Also added backport to 3.8 since passing invalid headers via e-mail can lead to DoS here.

a denial of service is exactly what can happen indeed. practically speaking, the widely usedimapclient indirectly calls this code, and as a result it unexpectedly crashes when listing mailbox contents if such a message is encountered.

@ambvambv merged commit989f6a3 intopython:mainAug 26, 2021
@miss-islington
Copy link
Contributor

Thanks@wbolster for the PR, and@ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7, 3.8, 3.9, 3.10.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-27972 is a backport of this pull request to the3.10 branch.

@bedevere-bot
Copy link

GH-27973 is a backport of this pull request to the3.9 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestAug 26, 2021
pythonGH-27946)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestAug 26, 2021
pythonGH-27946)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
@bedevere-bot
Copy link

GH-27974 is a backport of this pull request to the3.8 branch.

@bedevere-bot
Copy link

GH-27975 is a backport of this pull request to the3.7 branch.

@bedevere-bot
Copy link

GH-27976 is a backport of this pull request to the3.6 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestAug 26, 2021
pythonGH-27946)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestAug 26, 2021
pythonGH-27946)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
@wbolsterwbolster deleted the bpo45001-date-email-parsing-robustness branchAugust 26, 2021 15:03
miss-islington added a commit that referenced this pull requestAug 26, 2021
GH-27946)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
ambv pushed a commit that referenced this pull requestAug 26, 2021
GH-27946) (GH-27973)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
ambv pushed a commit that referenced this pull requestAug 26, 2021
GH-27946) (GH-27974)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
ned-deily pushed a commit that referenced this pull requestAug 30, 2021
GH-27946) (GH-27975)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
ned-deily pushed a commit that referenced this pull requestAug 30, 2021
GH-27946) (GH-27976)Various date parsing utilities in the email module, such asemail.utils.parsedate(), are supposed to gracefully handle invalidinput, typically by raising an appropriate exception or by returningNone.The internal email._parseaddr._parsedate_tz() helper used by some ofthese date parsing routines tries to be robust against malformed input,but unfortunately it can still crash ungracefully when a non-empty butwhitespace-only input is passed. This manifests as an unexpectedIndexError.In practice, this can happen when parsing an email with only a newlineinside a ‘Date:’ header, which unfortunately happens occasionally in thereal world.Here's a minimal example:    $ python    Python 3.9.6 (default, Jun 30 2021, 10:22:16)    [GCC 11.1.0] on linux    Type "help", "copyright", "credits" or "license" for more information.    >>> import email.utils    >>> email.utils.parsedate('foo')    >>> email.utils.parsedate(' ')    Traceback (most recent call last):      File "<stdin>", line 1, in <module>      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate        t = parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz        res = _parsedate_tz(data)      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz        if data[0].endswith(',') or data[0].lower() in _daynames:    IndexError: list index out of rangeThe fix is rather straight-forward: guard against empty lists, aftersplitting on whitespace, but before accessing the first element.(cherry picked from commit989f6a3)Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@wbolster@ambv@miss-islington@bedevere-bot@the-knights-who-say-ni

[8]ページ先頭

©2009-2025 Movatter.jp