Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.1k
bpo-36564: Fix infinite loop in email folding logic#12732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
I believe this will cause merge conflicts with#12020. I have subscribed to that PR and will rebase if it is merged first. |
Lib/email/_header_value_parser.py Outdated
chrome_len = len(encode_as) + 7 | ||
if (chrome_len + 1) >= maxlen: | ||
raise ValueError("Maximum length too short to fit any values") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I pickedValueError
here, but if there's a better error type let me know, I'm not super familiar with theemail
module.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We should probably useemail.errors.HeaderParseError
instead ofValueError
.
It would be nice if you could use single quotes for consistency in the module.
Also, the error message seems a bit vague given that it is not too short to fitany value, but too short to hold encoded word. An ascii value would be just fine with the line length). And,Maximum length
is a hard to make sense when coming from an exception message.max_line_length
is more greppable to me.
What do you think about this error?
max_line_length is too small to fit an encoded word
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
+1 forHeaderParseError
but@bitdancer has more up-to-date perspective than I might have.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
No, a ValueError would be more appropriate here, I think, because it is an error in thevalue of maxlen. This isn't aparsing error, it's arendering error.
Now, that said you might want to consider the fact that in_fold_mime_parameters
I deal with this issue by bumping maxlen to 78 rather than raising an error. I'm not sure that was the right choice, but whatever we do, it should probably be made consistent between the two cases. It is so sad that I never came back to fix that XXX comment I left myself :(
Note: I unfortunately am not too likely to see github comments, I get too much noise from that source...I just caught this one by chance. If you need my attention please post on the issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Also, the error message seems a bit vague given that it is not too short to fitany value, but too short to hold encoded word. An ascii value would be just fine with the line length). And,
Maximum length
is a hard to make sense when coming from an exception message.max_line_length
is more greppable to me.
I agree that "Maximum length" is not going to be a good error, switching tomax_line_length
is a good idea.
Regarding the "encoded word" situation, I guess when I wrote this I had assumed that the RFC chrome was added to every line regardless, but only realized that that wasn't the case through testing. I'm still not entirely clearwhy the encoding chrome is unnecessary for shorter words, but I think the suggested comments at least explainwhen the chrome becomes necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Overall, the changes look good to me. I have added some inline comments for error message and slight change to a comment.
Thanks@pganssle !
Lib/email/_header_value_parser.py Outdated
chrome_len = len(encode_as) + 7 | ||
if (chrome_len + 1) >= maxlen: | ||
raise ValueError("Maximum length too short to fit any values") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We should probably useemail.errors.HeaderParseError
instead ofValueError
.
It would be nice if you could use single quotes for consistency in the module.
Also, the error message seems a bit vague given that it is not too short to fitany value, but too short to hold encoded word. An ascii value would be just fine with the line length). And,Maximum length
is a hard to make sense when coming from an exception message.max_line_length
is more greppable to me.
What do you think about this error?
max_line_length is too small to fit an encoded word
.
Lib/test/test_email/test_policy.py Outdated
# required length for a line | ||
# Note: This is only triggered when there is a single word longer than | ||
# maxlen, hence the 1234567890 at the end of this whimsical subject |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Slight change to the comment to make it a bit more clear why this bug is trigerred when there is a single world longer thanmaxlen
.
# Note: This is only triggered when there is a single word longer than# max_line_length. This is because when we encounter a word longer than max_line_length,# it is broken down into encoded words to fit max_line_length. If the max_line_length isn't# large enough to even contain RFC 2047 chrome (`?=<charset>?q??=`), we fail.# Hence the 1234567890 at the end of this whimsical subject.
Just to make it clear that Header can have words longer than max line length, as long asmax_line_length > 7 + len(charset) + 1
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
(There's an extra space in your comment: "even contain"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
HeaderParseError
seems right to me too, but maybe@bitdancer has a different opinion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Actually, the phrase "even contain RFC 2047 chrome (?=<charset>?q??=
)" should probably be "even contain RFC 2047 chrome plus one character(?=<charset>?q?x?=
)"
Lib/test/test_email/test_policy.py Outdated
def test_short_maxlen_error(self): | ||
# RFC 2047 chrome takes up 7 characters, plus the length of the charset | ||
# name, so folding should fail if maxlen is lower than the minimum | ||
# required length for a line |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Missing period.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
See inlined comments.
bedevere-bot commentedMay 16, 2019
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Also@pganssle you may want to change the description, since the first point (The value being folded contains a single word (no spaces) longer than max_line_length) isn't true. |
Thanks for the reviews everyone. Just as an update, I am planning on making the requested changes soon, but I've just started a new job and I'm still working out some of the IP issues so I've been holding off on any non-trivial OSS work until that's sorted out. Shouldn't be too much longer. |
184072d
toaff66ad
CompareI have made the requested changes; please review again |
bedevere-bot commentedJul 14, 2019
Thanks for making the requested changes! @warsaw: please review the changes made to this pull request. |
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564
LGTM! @pganssle It is usually better to not force push and just keep adding more commits as your make changes your existing PR. It makes it a bit easy to remember what changes were made after the previous review, especially if it has been a while since last review :) |
Uh oh!
There was an error while loading.Please reload this page.
bedevere-bot commentedJul 16, 2019
When you're done making the requested changes, leave the comment: |
bedevere-bot commentedJul 16, 2019
GH-14797 is a backport of this pull request to the3.8 branch. |
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
bedevere-bot commentedJul 16, 2019
GH-14798 is a backport of this pull request to the3.7 branch. |
bedevere-bot commentedJul 16, 2019
GH-14799 is a backport of this pull request to the3.6 branch. |
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564(cherry picked from commitf69d5c6)Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564
As far as I can tell, this infinite loop would be triggered if:1. The value being folded contains a single word (no spaces) longer than max_line_length2. The max_line_length is shorter than the encoding's name + 9 characters.bpo-36564:https://bugs.python.org/issue36564
Uh oh!
There was an error while loading.Please reload this page.
As far as I can tell, this infinite loop would be triggered if:
max_line_length
characters.
bpo-36564:https://bugs.python.org/issue36564
https://bugs.python.org/issue36564