Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.4k
GH-78319: Stop sending the UTF8 marker when appending messages#107290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Open
arnt wants to merge1 commit intopython:mainChoose a base branch
base:main
Could not load branches
Branch not found:{{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline, and old review comments may become outdated.
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
…to a mailbox.The UTF8 marker is defined in RFC 6855 and tells the server that themessage being appended contains UTF8 addresses, an unencoded UTF8 subject,etc. However, if a client appends a message containing UTF8 addresses butwithout that marker, the bytes can only be parsed as UTF8 because that'sthe only RFC-compliant way to parse those bytes.RFC 6855 says clients MUST send the UTF8 marker.Due to an accidental discrepancy, RFC 9051 (IMAP4rev2) does not containthat marker. IMAP4rev2 was intended to be upwardly compatible with RFC6855, but this problem broke that. This has no ill effects, since themarker does not change the message's meaning.While investigating the problem, I noticed that Python uses the markerincorrectly: Python uses it to mark ALL messages if UTF8=ACCEPT supporthas been enabled, not just ones that contain UTF8 addresses.The best way forward appear to be using the syntax defined in RFC 9051 andpublishing a revision to RFC 6855, so this change modifies imaplib tomatch RFC 9051.FWIW JMAP is like IMAP4rev2 in this case; UTF8 is just there, without anymarker. Also, neither UTF8=ACCEPT, IMAP4rev2 or JMAP provide any way tolearn whether a message was stored with or without the marker.This quasi-accidentally solvespython#78319 by removing the case that broke.
Closed
bedevere-bot commentedJul 26, 2023
Most changes to Pythonrequire a NEWS entry. Please add it using theblurb_it web app or theblurb command-line tool. |
Author
arnt commentedJul 26, 2023
This seems small enough to not require a NEWS entry, I'd say. |
Author
arnt commentedJul 26, 2023
I submittedthe internet-draft to supersede RFC 6855 now and expect to get it to RFC fairly quickly. |
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
The UTF8 marker is defined in RFC 6855 and tells the server that the message being appended contains UTF8 addresses, an unencoded UTF8 subject, etc. However, if a client appends a message containing UTF8 addresses but without that marker, the bytes can only be parsed as UTF8 because that's the only RFC-compliant way to parse those bytes.
RFC 6855 says clients MUST send the UTF8 marker.
Due to an accidental discrepancy, RFC 9051 (IMAP4rev2) does not contain that marker. IMAP4rev2 was intended to be upwardly compatible with RFC 6855, but this problem broke that. This has no ill effects, since the marker does not change the message's meaning.
While investigating the problem, I noticed that Python uses the marker incorrectly: Python uses it to mark ALL messages if UTF8=ACCEPT support has been enabled, not just ones that contain UTF8 addresses.
The best way forward appear to be using the syntax defined in RFC 9051 and publishing a revision to RFC 6855, so this change modifies imaplib to match RFC 9051.
FWIW JMAP is like IMAP4rev2 in this case; UTF8 is just there, without any marker. Also, neither UTF8=ACCEPT, IMAP4rev2 or JMAP provide any way to learn whether a message was stored with or without the marker.
This quasi-accidentally solves#78319 by removing the case that broke.