Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-92081: Fix for email.generator.Generator with whitespace between encoded words.#92281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
warsaw merged 2 commits intopython:mainfromabadger:email-bytes-generator-breakage
May 20, 2024

Conversation

abadger
Copy link
Contributor

@abadgerabadger commentedMay 4, 2022
edited by bedevere-bot
Loading

email.generator.Generator currently does not handle whitespace between
encoded words correctly when the encoded words span multiple lines. The
current generator will create an encoded word for each line. If the end
of the line happens to correspond with the end real word in the
plaintext, the generator will place an unencoded space at the start of
the subsequent lines to represent the whitespace between the plaintext
words.

A compliant decoder will strip all the whitespace from between two
encoded words which leads to missing spaces in the round-tripped
output.

The fix for this is to make sure that whitespace between two encoded
words ends up inside of one or the other of the encoded words. This
fix places the space inside of the second encoded word.

Test case from#92081

@abadgerabadger requested a review froma team as acode ownerMay 4, 2022 00:57
@abadgerabadger marked this pull request as draftMay 4, 2022 00:57
@abadger
Copy link
ContributorAuthor

abadger commentedMay 5, 2022
edited
Loading

This is a work in progress because#92081 has one other issue that also needs to be fixed. Whitespace at the start of the Subject is being omitted as well. I'm leaning towards fixing that one in the decoder but I'm still reading through the rfcs to see if it has anything to say about that.

Note: the other issue has been fixed here as well and this is ready for review.

@abadgerabadgerforce-pushed theemail-bytes-generator-breakage branch frombc7a42d to80f5cfaCompareApril 26, 2023 22:03
@abadgerabadger marked this pull request as ready for reviewApril 26, 2023 22:24
@abadgerabadgerforce-pushed theemail-bytes-generator-breakage branch frombec90d8 to69b205dCompareApril 26, 2023 22:48
@@ -1628,7 +1629,7 @@ def test_address_display_names(self):
'Lôrem ipsum dôlôr sit amet, cônsectetuer adipiscing. '
'Suspendisse pôtenti. Aliquam nibh. Suspendisse pôtenti.',
'=?utf-8?q?L=C3=B4rem_ipsum_d=C3=B4l=C3=B4r_sit_amet=2C_c'
'=C3=B4nsectetuer?=\n =?utf-8?q?adipiscing=2E_Suspendisse'
'=C3=B4nsectetuer?=\n =?utf-8?q?_adipiscing=2E_Suspendisse'
Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Note, the data in the unittest was buggy. If you run the original output throughdecode_header() you'll find that it is missing the space betweencônsectetuer andadipiscing

@abadger
Copy link
ContributorAuthor

This fix is now ready to be reviewed.

alex-pobeditel-2004 reacted with heart emojierlend-aasland reacted with rocket emoji

@abadgerabadgerforce-pushed theemail-bytes-generator-breakage branch from69b205d to4104b8eCompareJuly 21, 2023 14:47
@abadger
Copy link
ContributorAuthor

Hey@warsaw , this fix is ready for review if you have some cycles to spare for thinking about email and the stdlib.

@alex-pobeditel-2004

@abadger I tested this fix on my project and it worked like a charm. Inexpressible thanks! Will use your patch until this will not be merged into CPython :)

email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are frompython#92081
@abadgerabadgerforce-pushed theemail-bytes-generator-breakage branch from4104b8e to5071b52CompareMay 20, 2024 17:40
@warsawwarsaw self-assigned thisMay 20, 2024
@warsawwarsaw added 3.12only security fixes 3.13bugs and security fixes needs backport to 3.12only security fixes needs backport to 3.13bugs and security fixes labelsMay 20, 2024
Copy link
Member

@warsawwarsaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for your contribution to Python!

erlend-aasland reacted with hooray emojierlend-aasland reacted with rocket emoji
@warsawwarsawenabled auto-merge (squash)May 20, 2024 18:07
@warsawwarsaw merged commita6fdb31 intopython:mainMay 20, 2024
@miss-islington-app
Copy link

Thanks@abadger for the PR, and@warsaw for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestMay 20, 2024
…ween encoded words. (pythonGH-92281)* Fix for email.generator.Generator with whitespace between encoded words.email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are frompythonGH-92081* Rename a variable so it's not confused with the final variable.(cherry picked from commita6fdb31)Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
@abadgerabadger deleted the email-bytes-generator-breakage branchMay 20, 2024 19:11
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestMay 20, 2024
…ween encoded words. (pythonGH-92281)* Fix for email.generator.Generator with whitespace between encoded words.email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are frompythonGH-92081* Rename a variable so it's not confused with the final variable.(cherry picked from commita6fdb31)Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
@bedevere-app
Copy link

GH-119245 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelMay 20, 2024
@bedevere-app
Copy link

GH-119246 is a backport of this pull request to the3.12 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.12only security fixes labelMay 20, 2024
warsaw pushed a commit that referenced this pull requestMay 20, 2024
…tween encoded words. (GH-92281) (#119245)* Fix for email.generator.Generator with whitespace between encoded words.email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are fromGH-92081* Rename a variable so it's not confused with the final variable.(cherry picked from commita6fdb31)Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
warsaw pushed a commit that referenced this pull requestMay 20, 2024
…tween encoded words. (GH-92281) (#119246)* Fix for email.generator.Generator with whitespace between encoded words.email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are fromGH-92081* Rename a variable so it's not confused with the final variable.(cherry picked from commita6fdb31)Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
estyxx pushed a commit to estyxx/cpython that referenced this pull requestJul 17, 2024
…ween encoded words. (python#92281)* Fix for email.generator.Generator with whitespace between encoded words.email.generator.Generator currently does not handle whitespace betweenencoded words correctly when the encoded words span multiple lines.  Thecurrent generator will create an encoded word for each line.  If the endof the line happens to correspond with the end real word in theplaintext, the generator will place an unencoded space at the start ofthe subsequent lines to represent the whitespace between the plaintextwords.A compliant decoder will strip all the whitespace from between twoencoded words which leads to missing spaces in the round-trippedoutput.The fix for this is to make sure that whitespace between two encodedwords ends up inside of one or the other of the encoded words.  Thisfix places the space inside of the second encoded word.A second problem happens with continuation lines.  A continuation line thatstarts with whitespace and is followed by a non-encoded word is fine becausethe newline between such continuation lines is defined as condensing toa single space character.  When the continuation line starts with whitespacefollowed by an encoded word, however, the RFCs specify that the word is runtogether with the encoded word on the previous line.  This is because normalwords are filded on syntactic breaks by encoded words are not.The solution to this is to add the whitespace to the start of the encoded wordon the continuation line.Test cases are frompython#92081* Rename a variable so it's not confused with the final variable.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@warsawwarsawwarsaw approved these changes

Assignees

@warsawwarsaw

Labels
3.12only security fixes3.13bugs and security fixestopic-email
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

5 participants
@abadger@alex-pobeditel-2004@warsaw@bedevere-bot@AlexWaygood

[8]ページ先頭

©2009-2025 Movatter.jp