➜
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2016-06-06 10:25 bytouilleMan, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| test_mail_2.py | touilleMan,2016-06-06 10:25 | |||
| test2.py | Zhang Wenbo,2017-06-12 07:31 | |||
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 3488 | merged | r.david.murray,2017-09-11 02:07 | |
| PR 4693 | merged | python-dev,2017-12-03 23:53 | |
| Messages (11) | |||
|---|---|---|---|
| msg267525 -(view) | Author: Leblond Emmanuel (touilleMan)* | Date: 2016-06-06 10:25 | |
The folding code for email headers crashes whenever it hits line _header_value_parser.py:432 given it try to call method `_fold_as_ew`.```(venv)➜ vigiechiro-api git:(in2p3) ✗ python test_mail.py Traceback (most recent call last): File "test_mail.py", line 36, in <module> mail.send(msg) File "/home/emmanuel/projects/vigiechiro-api/venv/lib/python3.5/site-packages/flask_mail.py", line 492, in send message.send(connection) File "/home/emmanuel/projects/vigiechiro-api/venv/lib/python3.5/site-packages/flask_mail.py", line 427, in send connection.send(self) File "/home/emmanuel/projects/vigiechiro-api/venv/lib/python3.5/site-packages/flask_mail.py", line 190, in send message.as_bytes() if PY3 else message.as_string(), File "/home/emmanuel/projects/vigiechiro-api/venv/lib/python3.5/site-packages/flask_mail.py", line 385, in as_bytes return self._message().as_bytes() File "/usr/lib/python3.5/email/message.py", line 179, in as_bytes g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.5/email/generator.py", line 115, in flatten self._write(msg) File "/usr/lib/python3.5/email/generator.py", line 195, in _write self._write_headers(msg) File "/usr/lib/python3.5/email/generator.py", line 422, in _write_headers self._fp.write(self.policy.fold_binary(h, v)) File "/usr/lib/python3.5/email/policy.py", line 190, in fold_binary folded = self._fold(name, value, refold_binary=self.cte_type=='7bit') File "/usr/lib/python3.5/email/policy.py", line 204, in _fold return self.header_factory(name, ''.join(lines)).fold(policy=self) File "/usr/lib/python3.5/email/headerregistry.py", line 255, in fold return header.fold(policy=policy) File "/usr/lib/python3.5/email/_header_value_parser.py", line 300, in fold self._fold(folded) File "/usr/lib/python3.5/email/_header_value_parser.py", line 1228, in _fold rest._fold(folded) File "/usr/lib/python3.5/email/_header_value_parser.py", line 432, in _fold part._fold_as_ew(folded)AttributeError: 'UnstructuredTokenList' object has no attribute '_fold_as_ew'> /usr/lib/python3.5/email/_header_value_parser.py(432)_fold()-> part._fold_as_ew(folded)```Given folding depends a lot on the processed value, this bug only trigger in very specific encoded headers (I've try to reproduce this bug with various values but only the one which originally made me find the bug worked so far).I've isolated the bug into a simple test case (see enclosed file).It seems this part of the code has been rewritten between Python2 and 3 (the file _header_value_parser.py doesn't exists in Python2). So unsurprisingly the bug doesn't trigger in Python2.Replacing `part._fold_as_ew(folded)` by `part._fold(folded)` seems to fix the bug, but given my poor knowledge of this code, this fix probably produce unwanted behaviors. | |||
| msg295751 -(view) | Author: Zhang Wenbo (Zhang Wenbo) | Date: 2017-06-12 07:31 | |
I think I find a case which can reproduct this bug: the UnstructuredTokenList has a ValueTerminal which is long enough(its encoded word is longer than policy.maxlen ) and not at the beginning.For examples, the following subjects can cause this bug:- 'Re: 我只是一个有那么一点点长的主题而已'- '回复: 回复:【北京大学-2017年招生信息公告】'So, the code can not fold a ValueTerminal correctly. Seems the method `_fold_as_ew` was designed to do this, but it is missing. | |||
| msg301456 -(view) | Author: Raphael Slinckx (rslinckx) | Date: 2017-09-06 08:43 | |
It happens regularly for emails with long header lines for me too, which makes the python3 email module quite unusable.A workaround which seems to help a bit is to set refold_source='none' on the email policy, although it seems to keep happening, less often.Is the fix proposed by Emmanuel a proper fix ? I'm not knowledgeable enough on this subject... | |||
| msg301481 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2017-09-06 16:49 | |
I started rewriting the header folder in response to this bug, because the root problem is a bit deeper than just _fold_as_ew being missing. I will probably work on it some more this week, but if I don't think I'm going to get it finished I'll look at putting in a fix that doesn't address the other problems but does address this one. | |||
| msg301796 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2017-09-10 00:31 | |
I managed to finish the rewrite, and it does fix this issue. I'll probably post the PR tomorrow. | |||
| msg307528 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2017-12-03 23:51 | |
New changeset85d5c18c9d83a1d54eecc4c2ad4dce63194107c6 by R. David Murray in branch 'master':bpo-27240 Rewrite the email header folding algorithm. (#3488)https://github.com/python/cpython/commit/85d5c18c9d83a1d54eecc4c2ad4dce63194107c6 | |||
| msg307530 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2017-12-04 00:46 | |
New changeseta87ba60fe56ae2ebe80ab9ada6d280a6a1f3d552 by R. David Murray (Miss Islington (bot)) in branch '3.6':bpo-27240 Rewrite the email header folding algorithm. (GH-3488) (#4693)https://github.com/python/cpython/commit/a87ba60fe56ae2ebe80ab9ada6d280a6a1f3d552 | |||
| msg307642 -(view) | Author: Berker Peksag (berker.peksag)* | Date: 2017-12-05 04:43 | |
All three examples in test2.py correctly folded (doesn't hit the ``except Exception as e:`` branch anymore) in current master. Can this issue be closed now? | |||
| msg307659 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2017-12-05 12:21 | |
Huh, I thought I had closed it. | |||
| msg330016 -(view) | Author: Vlad Temian (vtemian) | Date: 2018-11-17 00:58 | |
Is there any chance that this fix is gonna be ported to 3.5 as well? | |||
| msg330030 -(view) | Author: R. David Murray (r.david.murray)* | Date: 2018-11-17 14:54 | |
No. 3.5 is in security-fix-only mode. | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:32 | admin | set | github: 71427 |
| 2018-11-17 14:54:32 | r.david.murray | set | messages: +msg330030 |
| 2018-11-17 00:58:16 | vtemian | set | nosy: +vtemian messages: +msg330016 |
| 2017-12-05 12:21:29 | r.david.murray | set | status: open -> closed resolution: fixed messages: +msg307659 stage: patch review -> resolved |
| 2017-12-05 04:43:58 | berker.peksag | set | nosy: +berker.peksag messages: +msg307642 |
| 2017-12-04 00:46:25 | r.david.murray | set | messages: +msg307530 |
| 2017-12-03 23:53:07 | python-dev | set | pull_requests: +pull_request4605 |
| 2017-12-03 23:51:43 | r.david.murray | set | messages: +msg307528 |
| 2017-09-11 02:07:55 | r.david.murray | set | keywords: +patch stage: needs patch -> patch review pull_requests: +pull_request3478 |
| 2017-09-10 00:31:36 | r.david.murray | set | messages: +msg301796 |
| 2017-09-06 16:49:01 | r.david.murray | set | messages: +msg301481 versions: + Python 3.7, - Python 3.5 |
| 2017-09-06 08:43:39 | rslinckx | set | nosy: +rslinckx messages: +msg301456 |
| 2017-06-12 07:31:34 | Zhang Wenbo | set | files: +test2.py messages: +msg295751 |
| 2017-06-12 06:02:18 | Zhang Wenbo | set | nosy: +Zhang Wenbo |
| 2017-05-11 13:49:03 | archibald | set | nosy: +archibald |
| 2016-06-06 11:19:26 | SilentGhost | set | stage: needs patch type: behavior versions: - Python 3.2, Python 3.3, Python 3.4 |
| 2016-06-06 10:25:59 | touilleMan | create | |