
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2015-11-05 09:42 byvstinner, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| ast.patch | vstinner,2015-11-05 09:42 | review | ||
| Messages (7) | |||
|---|---|---|---|
| msg254093 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2015-11-05 09:42 | |
I wrote an AST optimizer which build AST from source code using ast.parse(), modify AST in place, and then compile the AST to bytecode.The problem is that the lineno and col_offset attributes of the "arg" AST node are uninitialized when AST is rebuild from Python objects: compile(ast_tree, ...). As a consequence, the compilation may fail because lineno or col_offset values are invalid.Attached patch enhancesParser/asdl_c.py to take "arg" attributes in account. It also updates the generated filesInclude/Python-ast.h andPython/Python-ast.c, and fix usage of the arg() function inPython/ast.c.It looks like only the "arg" AST node had attributes which were initialized (other AST nodes are handled with a different code path which already filled attributes).SeeParser/Python.asdl for the definition of the "arg" AST node.Note: Python 2.7 is not affected. In Python 2, function arguments are simple expressions, they don't have a dedicated type with lineno and col_offset attributes. "arg" was introduced in Python 3 with thePEP 3107 (function annotations). | |||
| msg254155 -(view) | Author: Stéphane Wirtel (matrixise)*![]() | Date: 2015-11-06 01:07 | |
Hi Victor,1. Result of the compilation -> success2. I have checked your patch, I think there is no problem.3. How can I test it? currently, everything is fine.For me, this patch is ok and working, but I need the point 3. | |||
| msg254202 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-11-06 16:02 | |
New changeset54d4290f0ec6 by Victor Stinner in branch 'default':Issue#25555: Fix parser and AST: fill lineno and col_offset of "arg" node whenhttps://hg.python.org/cpython/rev/54d4290f0ec6 | |||
| msg254203 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2015-11-06 16:05 | |
> 3. How can I test it? currently, everything is fine.Sorry, I don't know how to test it.To write my fix, I had to modify the compile() builtin function to be able to compile Python to AST, and then AST to Python. But without this fix I don't know how to test.I tried to revert the AST patch on my FAT Python to try to find which part of the code fails without the fix, but FAT Python doesn't compile anymore. I reverted the revert, and it still doesn't work!?For this bug, sorry, you have to trust me :-(I applied the fix to Python 3.6. I will wait a bit for buildbots before backporting to 3.5. | |||
| msg254204 -(view) | Author: Stéphane Wirtel (matrixise)*![]() | Date: 2015-11-06 16:07 | |
I trust you, I have reviewed the source code and it seems to be right. | |||
| msg254403 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-11-09 20:56 | |
New changeset2d8988f64cea by Victor Stinner in branch '3.5':Issue#25555: Fix parser and AST: fill lineno and col_offset of "arg" node whenhttps://hg.python.org/cpython/rev/2d8988f64cea | |||
| msg254409 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2015-11-09 21:14 | |
I backported the fix to Python 3.5. Since I'm not convinced that the bug can be triggered without modifying CPython (the bug only impacts FAT Python which installs a hook on the AST compiler), I don't want to backport it to Python 2.7 to avoid the risk of regression. | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:23 | admin | set | github: 69741 |
| 2015-11-09 21:14:10 | vstinner | set | status: open -> closed resolution: fixed messages: +msg254409 |
| 2015-11-09 20:56:15 | python-dev | set | messages: +msg254403 |
| 2015-11-06 16:07:51 | matrixise | set | messages: +msg254204 |
| 2015-11-06 16:05:58 | vstinner | set | messages: +msg254203 |
| 2015-11-06 16:02:35 | python-dev | set | nosy: +python-dev messages: +msg254202 |
| 2015-11-06 01:07:21 | matrixise | set | messages: +msg254155 |
| 2015-11-05 09:42:04 | vstinner | create | |