Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-107001: Add a stdlib decorator that copies/applies the ParameterSpec from one function to another#121693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
CarliJoy wants to merge12 commits intopython:main
base:main
Choose a base branch
Loading
fromCarliJoy:gh-107001

Conversation

CarliJoy
Copy link

@CarliJoyCarliJoy commentedJul 13, 2024
edited
Loading

Add a stdlib decorator that copies/applies the ParameterSpec from one function to another.

All information can be found inthe related issue

Note for review for JelleZijlstra and AlexWaygood (and maybe the typing council)


📚 Documentation preview 📚:https://cpython-previews--121693.org.readthedocs.build/en/121693/library/typing.html#typing.copy_func_params

vergenzt reacted with thumbs up emojicyberw, mcmanustfj, bellini666, and laggron42 reacted with heart emoji
@ghost
Copy link

ghost commentedJul 13, 2024
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@JelleZijlstra
Copy link
Member

Since this decorator would need to be handled specially by type checkers, let's not add it to CPython until there is a corresponding change tothe typing spec.

@CarliJoy
Copy link
Author

Since this decorator would need to be handled specially by type checkers, let's not add it to CPython until there is a corresponding change tothe typing spec.

What are you referring to as "handled specially"?
Do you mean that checking that the signature of the decorated and source function shall be checked "by specification" by the type checkers?
Otherwise it already works with the existing type system without any changes to the type checkers (see linked SO/MyPy Play in related Issue)

I can create a PR for the Spec as well.

Independent of this I still would be interested if the name is okay and if I should includeskip_first (see PR description).

@hugovk
Copy link
Member

Let's convert to draft for now.

@hugovkhugovk marked this pull request as draftJuly 13, 2024 15:02
@JelleZijlstra
Copy link
Member

Otherwise it already works with the existing type system without any changes to the type checkers (see linked SO/MyPy Play in related Issue)

You're right, I missed this, sorry. If this signature can be expressed in the existing type system, it doesn't need to be specified explicitly.

Copy link
Member

@JelleZijlstraJelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If this functiondoesn't need special handling from type checkers, what is the motivation for adding it totyping?

We have a few existing helpers intyping that are not special forms (e.g.,assert_never,AnyStr), but I don't know if this proposed addition is quite as pervasively useful as the existing ones.

As an alternative, have you considered contributing this function tohttps://github.com/hauntsaninja/useful_types/?

hauntsaninja reacted with thumbs up emoji
Lib/typing.py Outdated
def upstream_func(a: int, b: float, *, double: bool = False) -> float:
...

@copy_kwargs(upstream_func)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The namecopy_kwargs implies that it does something to only the kwargs and not the args, but that's not the case; it copies both.

I would use@copy_signature.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hi,

sorry haven't had time to work on this in the meanwhile.
I agree thatcopy_kwargs is not correct but neither iscopy_signature. As in my understanding a signature includes also the return value. And we explicitly do not copy these.

After some thinking I came up with:
copy_func_params.

This also opens up the possibility to create acopy_meth_params.
Thecopy_meth_params is the same ascopy_func_params but keeps theself (orcls) argument.

@CarliJoy
Copy link
Author

If this functiondoesn't need special handling from type checkers, what is the motivation for adding it totyping?

We have a few existing helpers intyping that are not special forms (e.g.,assert_never,AnyStr), but I don't know if this proposed addition is quite as pervasively useful as the existing ones.

As an alternative, have you considered contributing this function tohttps://github.com/hauntsaninja/useful_types/?

The function doesn't currentlyrequire type checkers to add any special support. However, introducing it in the typing module could pave the way for future enhancements, where type checkers or static analysis toolsmight validate that the parameters of two functions actually match. This would allow Python projects to ensure that signature changes—whether in their own code or due to third-party library updates—are caught early, preventing mismatches that could otherwise lead to subtle bugs.

While contributing this function to a separate project, likeuseful_types, is an option, it has limitations. Projects that suggest vendorization can complicate maintenance, especially in larger codebases. Moreover, this decorator addresses a common enough use case that it warrants inclusion in the standard library, ensuring both accessibility and support across Python versions.

@JelleZijlstra
Copy link
Member

While contributing this function to a separate project, like useful_types, is an option, it has limitations. Projects that suggest vendorization can complicate maintenance, especially in larger codebases. Moreover, this decorator addresses a common enough use case that it warrants inclusion in the standard library, ensuring both accessibility and support across Python versions.

Then I'd suggest you contribute it to useful_types first, and only add it to the standard library after it has actually proven to be widely useful.

Note that anything we add now will only be released with Python 3.14, in October 2025 (though we'd add it to typing-extensions earlier).

AlexWaygood and hauntsaninja reacted with thumbs up emoji

@CarliJoy
Copy link
Author

Then I'd suggest you contribute it to useful_types first, and only add it to the standard library after it has actually proven to be widely useful.

Actually, I believe this decorator needs the visibility only the standard library can provide.

The purpose of this PR is to make typing in Python both easier and more robust. Extending functions and methods is common across all experience levels, but creating this kind of decorator is challenging—I found it difficult to arrive at the solution myself. The number of threads on discuss.python.org, along with the Stack Overflow issue referenced in#107001, demonstrates that there’s a real demand and that others are also struggling.

Note that anything we add now will only be released with Python 3.14, in October 2025 (though we'd add it to typing-extensions earlier).

Adding it now means it will already be visible in the Python 3.14 documentation, making it easy to find and use as an example.

I’m not in favor of placing this in a third-party library, as it would limit additional ideas I have for this decorator. For example, as noted in the documentation, using it incorrectly can actually reduce type safety. I’m already considering ways to support tools likeruff in checking that decorated functions correctly include*args and**kwargs.

In the longer term, I’d even like to explore extending the Python type system itself to allow checking if a source function’s call signature is compatible with a decorated function.

Adding this to a third-party library would impede this progress and keep the solution obscure.

I understand every addition has maintenance costs, but the actual code is just six lines returning the function. So what’s the reason for being so reluctant to add it?

In the end, my goal is to contribute to a Python ecosystem with fewer untyped*args and**kwargs that neither I nor my IDE or type checker can fully interpret. I hope that including this decorator is a small but meaningful step in that direction.

sscherfke, makukha, cyberw, and vergenzt reacted with thumbs up emoji

@ghost
Copy link

ghost commentedOct 28, 2024
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

@CarliJoy
Copy link
Author

@JelleZijlstra Happy New year.
Did you have some time to look at my arguments?

@JelleZijlstra
Copy link
Member

Your statements about "additional ideas" and "extending the Python type system" make me even more reluctant to add this function now. Putting something in the standard library effectively means freezing it, and any future additions or tweaks have to overcome a large compatibility barrier.

And if nobody is willing to contribute this feature to a third-party library like useful-types, that again makes me question how commonly useful the feature is. The standard library isn't the place to incubate new features.

AlexWaygood and hauntsaninja reacted with thumbs up emoji

@cyberw
Copy link

cyberw commentedJan 11, 2025
edited
Loading

that again makes me question how commonly useful the feature is.

Let me just chime in here. I found this PR while looking for a feature such as this. My project,Locust, subclassesrequests.Session and overrides a few methods to add logging/tracking to each HTTP request, sometimes with identical signature, sometimes adding a few additional keyword arguments. Requests has a lot methods (get, post, put, etc) that support the same parameters as a more genericrequest()-method. Being able to copy the method signatures from that class to further wrappers in subclasses is very convenient (and I've been able to successfully use a copy of the code in this PR).

Most importantly, I think there is overwhelming evidence out there that a feature such as this is frequently sought, with no really good answers:
https://stackoverflow.com/questions/59717828/copy-type-signature-from-another-function
https://stackoverflow.com/questions/65954587/copying-function-signature-in-a-method
https://stackoverflow.com/questions/42420810/copy-signature-forward-all-arguments-from-wrapper-function
https://stackoverflow.com/questions/35512112/how-can-i-copy-a-python-functions-signature-and-define-a-new-function
https://www.reddit.com/r/learnpython/comments/sksy85/copy_type_signature_of_a_parameter_of_another/
https://discuss.python.org/t/copying-signature-of-init-to-a-mixin-class-method/58907/4

(that being said, personally I would have no problem using this feature just because it wasn't in the stdlib, I just want itsomewhere :)

@CarliJoy
Copy link
Author

CarliJoy commentedJan 13, 2025
edited
Loading

Thanks for chiming in, cyberw. I didn't find all the results you were looking for even so I looked quite intensively in the past.

The sole reason for this MR is to make thealready existing way to copy function parameters from one function to anotherknown andeasily usable.

This PR achieves both goals.

As documented, there is a possible issue with this decorator: if the copied call signature is incompatible with the decorated function's call signature, it can lead to aRuntime TypeError that type checkers won’t catch 💣. Essentially, the decorated function’s signature is overwritten.[Details]

My "additional ideas" about extending the Python type system aim to assist user to prevent this.

"Putting something in the standard library effectively means freezing it, and any future additions or tweaks have to overcome a large compatibility barrier."

I didnot propose changing the stdlib implementation in future here. Instead, I have two ideas to address the issue above,:

  1. Short-term: Implement a Ruff rule to ensure all decorated functions use*args and**kwargs, reducing the risk of runtime type errors (as noted in the documentation). The stdlib "frozen" state is desired here.
  2. Long-term/Maybe: Advocate for a change to the typing specification, enabling type checkers to validate the compatibility of the copied call signature with the decorated function.

Including this inuseful-types wouldn’t make it easier to find or use¹. I’d rather keep it onStackOverflow, where users can also learn about the potential pitfalls..

Never the less: Both options make it equally hard to create a Ruff rule to help users avoid the issue.

¹ useful-types does not have a documentation and is rather unknown.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra left review comments

@hugovkhugovkhugovk left review comments

@AlexWaygoodAlexWaygoodAwaiting requested review from AlexWaygoodAlexWaygood is a code owner

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@CarliJoy@JelleZijlstra@hugovk@cyberw

[8]ページ先頭

©2009-2025 Movatter.jp