Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-131927: Prevent emitting optimizer warnings twice in the REPL#131993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
iritkatriel merged 12 commits intopython:mainfromtomasr8:repl-warnings
Apr 12, 2025

Conversation

tomasr8
Copy link
Member

@tomasr8tomasr8 commentedApr 1, 2025
edited by bedevere-appbot
Loading

Fix based on the idea from@iritkatriel:#131927 (comment)

Yet another option is to leave ast_opt alone, and handle this in the repl. So the repl would save the warnings from the ast construction and then filter out duplicated warnings from the compilation call.

PEP 765 is new in 3.14 so I think we can skip a news entry?

incomplete_input=False,
)
with warnings.catch_warnings(record=True) as all_warnings:
warnings.simplefilter("always")
Copy link
Member

@iritkatrieliritkatrielApr 1, 2025
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can we use "default" instead of "always", and just one context manager that spans both compiler calls? I think that will do the de-duplication, if I understand the doc correctly.

https://docs.python.org/3/library/warnings.html#the-warnings-filter

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

From what I can tell it does not work in this case. The syntax warning uses_PyErr_EmitSyntaxWarning which in turn callsPyErr_WarnExplicitObject and sets theregistry (last argument) toNULL. Theregistry is used to dedup the warnings so when it's not used, all warnings are emitted regardless of the filter setting.

Here's an example that shows that:

importwarningsdef_compile():compile('1 is 1','input','eval')withwarnings.catch_warnings():warnings.simplefilter("default")_compile()_compile()

This is probably not something we want to change, so doing a manual deduplication in pyrepl seems like the way to go. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe we can pass a registry?

Copy link
MemberAuthor

@tomasr8tomasr8Apr 2, 2025
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe, though the registry would have to persist between separate invocations ofcompile. I'll check if there's a nice way to do that.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Ok so I spent some time going over the warning code and this is my understanding so far. The warning registries are dictionaries that are inserted as__warningregistry__ into the globals of a Python frame. Which frame this is, is controlled by the stack level. The registries are created on demand insetup_context:

staticint
setup_context(Py_ssize_tstack_level,
PyTupleObject*skip_file_prefixes,
PyObject**filename,int*lineno,
PyObject**module,PyObject**registry)
{

This function is pretty complex and depends on other pieces in the warnings module so we cannot construct a registry manually in_PyErr_EmitSyntaxWarning (which is the one responsible for emitting warnings from ast_opt and currently sets the registry toNULL):

cpython/Python/errors.c

Lines 1909 to 1910 in37bc386

if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning,msg,filename,
lineno,NULL,NULL)<0)

One thing we could do, though I don't know how desirable it is, is to add a new private function akin toPyErr_WarnExplicitObject which sets up the registry for us. Basically something like this:

int- PyErr_WarnExplicitObject(PyObject *category, PyObject *message,-                          PyObject *filename, int lineno,-                          PyObject *module, PyObject *registry)+ _PyErr_WarnExplicitObjectRegistry(PyObject *category, PyObject *message,+                                  PyObject *filename, int lineno){    PyObject *res;    if (category == NULL)        category = PyExc_RuntimeWarning;    PyThreadState *tstate = get_current_tstate();    if (tstate == NULL) {        return -1;    }+    PyObject *unused_filename, *module, *registry;+    int unused_lineno;++    if (!setup_context(1, NULL,+                       &unused_filename, &unused_lineno, &module, &registry))+        return -1;    warnings_lock(tstate->interp);    res = warn_explicit(tstate, category, message, filename, lineno,                        module, registry, NULL, NULL);    warnings_unlock(tstate->interp);    if (res == NULL)        return -1;    Py_DECREF(res);    return 0;}

This seems to eliminate the duplicated syntax warnings in the REPL. What do you think about this approach?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I like this. Can_PyErr_WarnExplicitObjectRegistry just create a registry and callPyErr_WarnExplicitObject with it?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think so, basically something like this:

int_PyErr_WarnExplicitObjectRegistry(PyObject*category,PyObject*message,PyObject*filename,intlineno){PyObject*unused_filename,*module,*registry;intunused_lineno;intstack_level=1;if (!setup_context(stack_level,NULL,&unused_filename,&unused_lineno,&module,&registry))return-1;intrc=PyErr_WarnExplicitObject(category,message,filename,lineno,module,registry);Py_DECREF(module);returnrc;}

I'll update the PR to see if everything still works with this change.

@iritkatriel
Copy link
Member

tests are failing.

@tomasr8
Copy link
MemberAuthor

tests are failing.

Yep, I suspect it's becausesetup_context fails which then causes_PyErr_EmitSyntaxWarning to raise instead of emit a warning (I should probably change that). Though I'm not able to reproduce this locally yet.

@python-cla-bot
Copy link

All commit authors signed the Contributor License Agreement.

CLA signed

@iritkatriel
Copy link
Member

LGTM.@serhiy-storchaka what do you think?

@tomasr8
Copy link
MemberAuthor

Ok it seems some other test modified the warning filters so I just needed to addwarnings.simplefilter("default") to the test.

I have another question though, regarding thesetup_context function. It can currently fail but I'm not sure what is the best way to handle that? Should we raise a SystemError or maybe just fall back to emitting a warning without constructing the registry (and silently ignore the failure)? Or just do what we do now and let it raise a SyntaxError (instead of emitting a warning)?

warnings.simplefilter("default")
console.runsource(code)

count = sum("'return' in a 'finally' block" in str(w.message)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

You can use.count() here.

"'return' in a 'finally' block" may be an error in future versions. Maybe use some more long living warnings, likeassert(1, 'message') or1 is 1? BTW, they are currently emitted 3 times, not 2.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

You can use .count() here.

Hmm I don't think I can, unless I want to match the warning message exactly?

"'return' in a 'finally' block" may be an error in future versions. Maybe use some more long living warnings, like assert(1, 'message') or 1 is 1?

I can't if I want to test the new repl. The PEP 765 warning is the only one emitted from the ast optimizer which is needed to trigger this behaviour in the repl. However this does change the behaviour of all warnings that use_PyErr_EmitSyntaxWarning so I also added tests totest_compile for it.

""")

with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("default")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

What happens if set it to "always"?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It emits twice, I added a test totest_compile for it

Copy link
Member

@serhiy-storchakaserhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM. 👍

Brett2148 reacted with laugh emojitomasr8 reacted with rocket emoji
@serhiy-storchaka
Copy link
Member

Please add also acompile() test for a warning emitted in thefinally block. Currently it is emitted twice because the code in thefinally block is complied twice (for normal execution and for exception handling).

Brett2148 reacted with laugh emoji

@iritkatrieliritkatriel merged commit3d08c8a intopython:mainApr 12, 2025
41 checks passed
@iritkatriel
Copy link
Member

Let's create a new issue for that.

tomasr8 reacted with thumbs up emoji

1 similar comment
@iritkatriel
Copy link
Member

Let's create a new issue for that.

@tomasr8tomasr8 deleted the repl-warnings branchApril 12, 2025 11:30
@tomasr8
Copy link
MemberAuthor

Please add also acompile() test for a warning emitted in thefinally block. Currently it is emitted twice because the code in thefinally block is complied twice (for normal execution and for exception handling).

PR is here:#132436

Brett2148 reacted with laugh emoji

@tomasr8tomasr8 added the needs backport to 3.13bugs and security fixes labelApr 12, 2025
@miss-islington-app
Copy link

Thanks@tomasr8 for the PR, and@iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Sorry,@tomasr8 and@iritkatriel, I could not cleanly backport this to3.13 due to a conflict.
Please backport usingcherry_picker on command line.

cherry_picker 3d08c8ad20dfabd4864be139cd9c2eb5602ccdfe 3.13

@tomasr8
Copy link
MemberAuthor

I'll take care of the backport

tomasr8 added a commit to tomasr8/cpython that referenced this pull requestApr 13, 2025
…the REPL (pythonGH-131993)(cherry picked from commit3d08c8a)Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
tomasr8 added a commit to tomasr8/cpython that referenced this pull requestApr 13, 2025
…the REPL (pythonGH-131993)(cherry picked from commit3d08c8a)Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
tomasr8 added a commit to tomasr8/cpython that referenced this pull requestApr 13, 2025
…the REPL (pythonGH-131993)(cherry picked from commit3d08c8a)Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
@bedevere-app
Copy link

GH-132463 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelApr 13, 2025
@bedevere-app
Copy link

GH-132463 is a backport of this pull request to the3.13 branch.

1 similar comment
@bedevere-app
Copy link

GH-132463 is a backport of this pull request to the3.13 branch.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@iritkatrieliritkatrieliritkatriel approved these changes

@serhiy-storchakaserhiy-storchakaserhiy-storchaka approved these changes

@pablogsalpablogsalAwaiting requested review from pablogsalpablogsal is a code owner

@lysnikolaoulysnikolaouAwaiting requested review from lysnikolaoulysnikolaou is a code owner

@ambvambvAwaiting requested review from ambvambv is a code owner

Assignees

@iritkatrieliritkatriel

Labels
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@tomasr8@iritkatriel@serhiy-storchaka

[8]ページ先頭

©2009-2025 Movatter.jp