Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-137530: generate an __annotate__ function for dataclasses __init__#137711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
encukou merged 20 commits intopython:mainfromDavidCEllis:fix-init-annotations-alt
Nov 10, 2025

Conversation

@DavidCEllis
Copy link
Contributor

@DavidCEllisDavidCEllis commentedAug 13, 2025
edited by bedevere-appbot
Loading

This is one approach to resolving#137530

It generates a new__annotate__ function to attach to the__init__ method and removes the in-line annotations that are now unused.

It's possible to keep the original source annotations if necessary, but they provide a second possibly incorrect source of the information and they will probably also cause additional issues with#135228 as they may keep the original class around when ForwardRef values are generated.

The generated__annotate__ will also keep these references and need to be regenerated. I can extend this or follow up to handle these references but I didn't want to do too much in one PR.

elifisinstance(v,annotationlib.ForwardRef):
string_annos[k]=v.evaluate(format=annotationlib.Format.STRING)
else:
string_annos[k]=annotationlib.type_repr(v)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is a bit unfortunate because we could get better STRING annotations by calling with the STRING format on the original class (and its base classes).

An approach to do this would be to make__init__.__annotate__ delegate to callingget_annotations() on the class itself, and on any base classes that are contributing fields, with the appropriate format, then processing the result to add"return": None and filter out any fields that don't correspond to__init__ parameters.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I did think about that, but wasn't certain about the extra complexity, but I've since noticed that if there's a hint thatcontains a ForwardRef (likelist[undefined]) the result leaks all the ForwardRef details.

I've added a test for that example to demonstrate why this is necessary.

I've taken a slightly different approach, in that I'm gathering all inherited annotations and using them to update the values from the fields. The original source annotation logic this replaces wasn't__init__ specific so I've tried to keep the__annotate__ generator non-specific too.


There are additional commits as this change meant that__annotate__ now (always) has a reference to the original class which broke#135228 - this now also includes an additional test and fix for the issue I brought up there as I already needed to update the fields for this.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I also unintentionally discovered that the__annotate__ functions CPython generates for methods have an incorrect__qualname__ while fixing the__qualname__ for the function generated here.

@DavidCEllis
Copy link
ContributorAuthor

On doing some other work, I think the logic forSTRING is also needed forFORWARDREF. I'm not sure how to resolveVALUE.

Problem for both:

fromannotationlibimportget_annotationsfromdataclassesimportdataclass@dataclassclassExample:a:list[defined_late]defined_late=intprint(get_annotations(Example.__init__))
{'a': list[ForwardRef('defined_late', is_class=True, owner=<class '__main__.Example'>)], 'return': None}

ForFormat.FORWARDREF this is resolvable in the same way asSTRING. However forVALUE it doesn't quite work out to do the same thing, as the annotations on the class could contain a name that doesn't exist yet - but ifinit=False that name may not be required for the__init__ annotations.

@DavidCEllis
Copy link
ContributorAuthor

Ah, while this is resolvable forFORWARDREF - it requiresVALUE to fail orget_annotations will useVALUE instead anyway.

I'm leaning towards makingVALUE fail if any of the class annotations fail withVALUE. I can't currently think of a clean way otherwise to both makeFORWARDREF evaluate as far as possibleand not leakForwardRef values intoVALUE annotations.

Also split out the single large test into multiple smaller tests
@DavidCEllis
Copy link
ContributorAuthor

DavidCEllis commentedAug 15, 2025
edited
Loading

Thoughts:

  • With the change to collect class annotations, all of thevalues from theannotations argument are no longer used apart from the return type.
    • Perhaps_make_annotate_function should take(cls, annotation_fields, return_type) instead of(cls, annotations), where annotation_fields is just a list of relevant field names?
    • _FuncBuilder.add_fn would regainreturn_type and replaceannotations withannotation_fields
  • I'd like to make theVALUE annotations work intest_init_false_forwardref but I don't currently see a clean way to do so.

Edit: I realised changing the signature would make_add_slots simpler and is probably worthwhile.

Use `__class__` as the first argument name so `_update_func_cell_for__class__` can update it.
@DavidCEllis
Copy link
ContributorAuthor

Ok, I'm going to try to leave this alone for some review.

The only functional1 thing Iknow I'm not completely happy with is this example from the tests2:

fromdataclassesimportdataclass,fieldfromannotationlibimportget_annotations@dataclassclassF:not_in_init:list[undefined]=field(init=False,default=None)in_init:intannos=annotationlib.get_annotations(F.__init__)# NameError

I don't have a good approach for this. I don't think you can useFORWARDREF annotations as they are without potentially leaking actual ForwardRefs?

Footnotes

  1. The first parameter of_make_annotate_function being named__class__ in order to reuse the cell update function could maybe be cleaner.

  2. Ignore the incorrect default type, initially the annotation had| None and I forgot to remove the default when I removed it.

@DavidCEllis
Copy link
ContributorAuthor

The ideal solution I would like for this would require an additional feature fromannotationlib to provide completely unevaluated references.

Then a solution more like the original version of this PR could work.

Copy link
Member

@JelleZijlstraJelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks good, except for the small suggestions above.

@JelleZijlstraJelleZijlstra added the needs backport to 3.14bugs and security fixes labelOct 21, 2025
@JelleZijlstra
Copy link
Member

You'll also need a NEWS entry. Since we're planning to backport to 3.14 to fix the release blocker, I think it's also worth mentioning in What's New as this will be a fairly big change in 3.14.1.

@encukou
Copy link
Member

Looks like the What's New entry all that's left here.@DavidCEllis, do you want to write one? Or I can do it to help get this in.

@DavidCEllis
Copy link
ContributorAuthor

I'm happy for you to do it. It's not something I've needed to do before so I don't actually know where it would need to go.

encukou reacted with thumbs up emoji

@encukou
Copy link
Member

It would go here. For the backport, there'd be a “Notable changes” section at the end, likefor 3.13.


@JelleZijlstra ... um, are you sure this is worth it? From anoutside point of view this looks like a straightforward regression fix.

DavidCEllis reacted with thumbs up emoji

@encukou
Copy link
Member

ping@JelleZijlstra

I plan to merge Monday or so if I don't hear back, to give enough time for a backport.

@encukouencukou merged commit12837c6 intopython:mainNov 10, 2025
46 checks passed
@miss-islington-app
Copy link

Thanks@DavidCEllis for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Sorry,@DavidCEllis and@encukou, I could not cleanly backport this to3.14 due to a conflict.
Please backport usingcherry_picker on command line.

cherry_picker 12837c63635559873a5abddf511d38456d69617b 3.14

JelleZijlstra pushed a commit to JelleZijlstra/cpython that referenced this pull requestNov 10, 2025
…sses __init__ (pythonGH-137711)(cherry picked from commit12837c6)Co-authored-by: David Ellis <ducksual@gmail.com>
@bedevere-app
Copy link

GH-141352 is a backport of this pull request to the3.14 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.14bugs and security fixes labelNov 10, 2025
@JelleZijlstra
Copy link
Member

Sorry for not responding here earlier, haven't been paying enough attention to GitHub.

um, are you sure this is worth it? From an outside point of view this looks like a straightforward regression fix.

Fair enough, my thinking was that this significantly changes the annotations people will see if they introspect dataclasses init functions. But our What's New documents don't seem to even have sections for changes in bugfix releases recently, so maybe it's not worth it.

JelleZijlstra added a commit that referenced this pull requestNov 10, 2025
…_init__ (GH-137711) (#141352)(cherry picked from commit12837c6)Co-authored-by: David Ellis <ducksual@gmail.com>
@encukou
Copy link
Member

But our What's New documents don't seem to even have sections for changes in bugfix releases recently, so maybe it's not worth it.

They do, at the end -- for example,https://docs.python.org/3.12/whatsnew/3.12.html#notable-changes-in-3-12-4
It's mainly used for security fixes backported to all active versions, though.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra left review comments

@ericvsmithericvsmithAwaiting requested review from ericvsmithericvsmith is a code owner

@AA-TurnerAA-TurnerAwaiting requested review from AA-TurnerAA-Turner is a code owner

Assignees

@encukouencukou

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DavidCEllis@JelleZijlstra@encukou

[8]ページ先頭

©2009-2025 Movatter.jp