Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-86682: Adds sys._getframemodulename as a lightweight alternative to _getframe#99520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
zooba merged 15 commits intopython:mainfromzooba:gh-86682
Jan 13, 2023

Conversation

@zooba
Copy link
Member

@zoobazooba commentedNov 15, 2022
edited by bedevere-bot
Loading

Replaces _getframe calls in collections, doctest, enum, and typing modules

corona10 reacted with thumbs up emoji
…ame calls in collections, doctest, enum, and typing modules
@zoobazooba changed the titlegh-86682: Adds sys._get_calling_module_namegh-86682: Adds sys._getcallerNov 16, 2022
@zoobazooba changed the titlegh-86682: Adds sys._getcallergh-86682: Adds sys._getcaller as a lightweight alternative to _getframeNov 16, 2022
Copy link
Member

@JelleZijlstraJelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The typing change is fine, thanks for picking this up!

I might like your original version (sys._get_calling_module_name) better. It exposes less data, so it's easier to keep the interface if interpreter internals change.

@AlexWaygoodAlexWaygood removed their request for reviewNovember 16, 2022 14:55
@zooba
Copy link
MemberAuthor

Yeah, turns out we can't use__module__ reliably here, so I'm inclined to change it back and make it only return the module name (rather than encoding use of__globals__ on the caller's side).

@zoobazooba changed the titlegh-86682: Adds sys._getcaller as a lightweight alternative to _getframegh-86682: Adds sys._getcallingmodule as a lightweight alternative to _getframeNov 17, 2022
Copy link
Member

@gvanrossumgvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

IMO the news file should mention the new function too.

I’d like a review by@markshannon .

I’m surprised the callingfunction isn’t obtainable, but the use cases are there, so no complaints about that.

@zooba
Copy link
MemberAuthor

I'd like a review from Mark, too.

The calling function is obtainable, even more easily than the module name. But in every case we already use this, we'd then have to go tofunc.__globals__['__name__'] and handle the cases where these are not consistent (__module__ is often<module> and not the actual module name).

If there's a use case for both, I'll happily add both, but right now, we don't need the function object for anything.

Copy link
Member

@gvanrossumgvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Really need to get@markshannon to answer the question. But he’s on vacation this week.

@markshannon
Copy link
Member

I'm thinking aloud here, so bear with me...

I don't think that_getframemodulename is a good name.
Not that_getframemodulename is a bad name for this API, maybe its just that the API is a bit odd?

It feels to me that this API is doing two things, getting the frame, then the module name. Why not split the parts?

frame = sys._getframe(n)modulename = get_module_name(frame)

or, potentially

modulename = sys._getframe(n).f_modulename

Unfortunately this doesn't achieve thestated goal of removing (almost) all uses ofsys._getframe(n)

Which leads us back toadding a function to get the caller function.
Functions already have a__module__ attribute, so we get

modulename = sys.getcallerfunc(n).__module__

Which seems cleaner to me.

@markshannon
Copy link
Member

@zooba You state in an earlier comment that__module__ is not reliable.
It seems to me that we should fix that, rather than working around the bug.

@markshannon
Copy link
Member

My cursory scan of the code suggests that__module__ == "<module>" only occurs in the repl, doctests and other source-less code.

@markshannon
Copy link
Member

Even in the repl, it seems that functtions have__module__ == "main".

@zooba
Copy link
MemberAuthor

@zooba You state in an earlier comment that__module__ is not reliable.
It seems to me that we should fix that, rather than working around the bug.

I didn't trace down exactly why it wasn't being filled in correctly, but I'm willing to bet it's a much more complicated compatibility issue to change it.__module__ is essentially a public API, while_getframemodulename is not.

It's definitely a bit of an odd API though, I agree. That's why I happily tried returning the function object instead. Perhaps it was only doctests that didn't set it correctly?

Also keen on your thoughts on theFRAME_OWNED_BY_CSTACK flagin this comment,@markshannon. I think I've used it right, but couldn't convince myself that this is the right way to skip frames.

@markshannon
Copy link
Member

You need to check for invalid frames when walking the frame stack.
You might want to factor out the walking the stack insys._getframe()https://github.com/python/cpython/blob/main/Python/sysmodule.c#L1889

@markshannon
Copy link
Member

OOI, what tests fail if you rely on__module__?

@zooba
Copy link
MemberAuthor

I've been looking into the__module__ issue, and it basically seems that<module> functions don't have__module__ set, which means anything at the top-level of a module won't work (e.g. virtually every typing, enum and dataclass definition). And this is the primary reason we use this, because anything that we'll be able to unpickle later on is going to have to be an importable name.

The only reliable workaround is to get it fromf_globals, as the code currently does. Tracing through the code, I think it's probably best to getfunc_module first if it's available, as that should let code assign tofunc.__module__ and override what gets returned. Then we can fall back tof_globals['__module__'] if it's not on the function object (yet) and still match.

I added a better test to compare_getframe with_getframemodulename and updated the check to match what is used in_getframe (which looks like a fairly different condition, but I trust you).

@markshannon
Copy link
Member

markshannon commentedDec 3, 2022
edited
Loading

it basically seems that<module> functions don't have__module__ set

That sounds like a bug. Let's fix it rather than working around it.

@zooba
Copy link
MemberAuthor

Seems easy enough to fix, there's just a constructor that needs to pull__name__ out of globals.

Even so, I still don't see a reason why we'd return function objects. Our code either needs the module name, or else it needs actual frame information (e.g. current lineno). This is a private API, so if we need something different in the future we can always change it.

This is also why I made it not raise an error but returnNone, since everywhere we'd use it we would swap in a default value anyway (I was tempted to add adefault argument, but we'd want to replace explicitNone as well, so the semantics work better for us this way). We really never need the function object right now, at least not anywhere we don't also need the locals or lineno.

@zoobazooba changed the titlegh-86682: Adds sys._getcallingmodule as a lightweight alternative to _getframegh-86682: Adds sys._getframemodulename as a lightweight alternative to _getframeDec 5, 2022
@markshannon
Copy link
Member

Even so, I still don't see a reason why we'd return function objects.

Two main reasons:

  • It is a simpler, cleaner API. It makes introspecting the frame stack for profilers and the like (that don't care about the values of local variables) more efficient, and less of an auditing issue.
  • Because we can. Until 3.11, we couldn't implement it. I suspect that if we could have, some of use cases that you describe would have used it before now.

My feeling is thatsys.getcallerfunc(n) wouldnot need an audit hook. Would you agree?
Or is that an argument in favor ofsys._getframemodulename()?

@zooba
Copy link
MemberAuthor

My feeling is that sys.getcallerfunc(n) would not need an audit hook. Would you agree?
Or is that an argument in favor of sys._getframemodulename()?

I started writing a reply earlier making the argument that only returning the name wouldn't require a red flag (I think it still deserves an event, but it's far less of a risk than if random code is unexpectedly doing_getframe). But I stopped because I don't actually have a strong argument, just a gut feel.

The most interesting thing you can do with a function object is mess withfunc_globals, which is the same assys.modules[sys._getframemodulename()] anyway, and dict lookups don't get an audit event (though possibly we could special casesys.modules if needed... I'd do that as my own patch first before trying to upstream it).

But because we don't use it, it means instead ofsys._getframemodulename() or 'default', we'd be writinggetattr(sys._getframefunction(), '__module__', None) or 'default' everywhere. That, plus YAGNI, is my argument for only returning the name (or None).

@zooba
Copy link
MemberAuthor

If there are no other concerns raised I'd like to merge this sometime this week.

@zoobazooba merged commitb5d4347 intopython:mainJan 13, 2023
@zoobazooba deleted the gh-86682 branchJanuary 13, 2023 11:31
op->func_dict=NULL;
op->func_weakreflist=NULL;
op->func_module=NULL;
op->func_module=Py_XNewRef(PyDict_GetItem(op->func_globals,&_Py_ID(__name__)));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

PyDict_GetItem is broken by design and should not be used. See#106033.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra left review comments

@gvanrossumgvanrossumgvanrossum left review comments

@serhiy-storchakaserhiy-storchakaserhiy-storchaka left review comments

@warsawwarsawwarsaw approved these changes

@ethanfurmanethanfurmanAwaiting requested review from ethanfurmanethanfurman is a code owner

@Fidget-SpinnerFidget-SpinnerAwaiting requested review from Fidget-Spinner

@rhettingerrhettingerAwaiting requested review from rhettingerrhettinger is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zooba@markshannon@warsaw@JelleZijlstra@gvanrossum@serhiy-storchaka@bedevere-bot

[8]ページ先頭

©2009-2025 Movatter.jp