Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-102578: Optimise setting and deleting mutable attributes on non-dataclass subclasses of frozen dataclasses#102573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
ericvsmith merged 15 commits intopython:mainfromXuehaiPan:dataclasses-lookup
Mar 11, 2023

Conversation

@XuehaiPan
Copy link
Contributor

@XuehaiPanXuehaiPan commentedMar 10, 2023•
edited
Loading

Creatingdataclasses with argumentfrozen=True will automatically generate methods__setattr__ and__delattr__ in_frozen_get_del_attr.

This PR changestuple-based lookup toset-based lookup. Reduce the time complexity from$O(n)$ to$O(1)$.

In [1]:# tuple-basedIn [2]:%timeit'a'in ('a','b','c','d','e','f','g')9.91ns ±0.0982nsperloop (mean ±std.dev.of7runs,100,000,000loopseach)In [3]:%timeit'd'in ('a','b','c','d','e','f','g')33.2ns ±0.701nsperloop (mean ±std.dev.of7runs,10,000,000loopseach)In [4]:%timeit'g'in ('a','b','c','d','e','f','g')56.4ns ±0.818nsperloop (mean ±std.dev.of7runs,10,000,000loopseach)In [5]:# set-basedIn [6]:%timeit'a'in {'a','b','c','d','e','f','g'}11.3ns ±0.0723nsperloop (mean ±std.dev.of7runs,100,000,000loopseach)In [7]:%timeit'd'in {'a','b','c','d','e','f','g'}11ns ±0.106nsperloop (mean ±std.dev.of7runs,100,000,000loopseach)In [8]:%timeit'g'in {'a','b','c','d','e','f','g'}11.1ns ±0.126nsperloop (mean ±std.dev.of7runs,100,000,000loopseach)

A tiny benchmark script:

fromcontextlibimportsuppressfromdataclassesimportFrozenInstanceError,dataclass@dataclass(frozen=True)classFoo2:a:intb:intfoo2=Foo2(1,2)defbench2(inst):withsuppress(FrozenInstanceError):inst.a=0withsuppress(FrozenInstanceError):inst.b=0@dataclass(frozen=True)classFoo7:a:intb:intc:intd:inte:intf:intg:intfoo7=Foo7(1,2,3,4,5,6,7)defbench7(inst):withsuppress(FrozenInstanceError):inst.a=0withsuppress(FrozenInstanceError):inst.b=0withsuppress(FrozenInstanceError):inst.c=0withsuppress(FrozenInstanceError):inst.d=0withsuppress(FrozenInstanceError):inst.e=0withsuppress(FrozenInstanceError):inst.f=0withsuppress(FrozenInstanceError):inst.g=0classBar(Foo7):def__init__(self,a,b,c,d,e,f,g):super().__init__(a,b,c,d,e,f,g)self.baz=0defbench(inst):inst.baz=1

Result:

set-based lookup:

In [2]:%timeitbench2(foo2)1.08µs ±28.1nsperloop (mean ±std.dev.of7runs,1,000,000loopseach)In [3]:%timeitbench7(foo7)3.81µs ±20.3nsperloop (mean ±std.dev.of7runs,100,000loopseach)In [4]:%timeitbench(bar)249ns ±6.31nsperloop (mean ±std.dev.of7runs,1,000,000loopseach)

tuple-based lookup (original):

In [2]:%timeitbench2(foo2)1.15µs ±10.9nsperloop (mean ±std.dev.of7runs,1,000,000loopseach)In [3]:%timeitbench7(foo7)3.97µs ±15.7nsperloop (mean ±std.dev.of7runs,100,000loopseach)In [4]:%timeitbench(bar)269ns ±4.09nsperloop (mean ±std.dev.of7runs,1,000,000loopseach)

Theset-based is constantly faster than the old approach. And the theoretical time complexity is also smaller ($O(1)$ vs. $O(n)$).

Resolves#102578

@bedevere-bot
Copy link

Most changes to Pythonrequire a NEWS entry.

Please add it using theblurb_it web app or theblurb command-line tool.

@ghost
Copy link

ghost commentedMar 10, 2023•
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

@XuehaiPanXuehaiPan changed the titleUseset-based name lookup rather thantuples for frozen dataclassesgh-102573: Useset-based name lookup rather thantuples for frozen dataclassesMar 10, 2023
@AlexWaygood
Copy link
Member

AlexWaygood commentedMar 10, 2023•
edited
Loading

Theissue-number CI check is failing becausegh-102573 does not yet exist as a GitHub issue :)

You should create an issue to describe the change you're proposing here, and then link to it in the title of this PR.

@XuehaiPanXuehaiPan changed the titlegh-102573: Useset-based name lookup rather thantuples for frozen dataclassesgh-102578: Useset-based name lookup rather thantuples for frozen dataclassesMar 10, 2023
@XuehaiPan
Copy link
ContributorAuthor

Opened a linked issue and added some benchmark results.

AlexWaygood reacted with thumbs up emoji

XuehaiPanand others added2 commitsMarch 10, 2023 22:08
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@AlexWaygoodAlexWaygood added performancePerformance or resource usage stdlibStandard Library Python modules in the Lib/ directory labelsMar 10, 2023
@AlexWaygood
Copy link
Member

Looks like the CLA check has started failing with the latest commit -- if you're using two email address, you may have to sign it with both email addresses, unfortunately :(

XuehaiPan reacted with heart emoji

@XuehaiPan
Copy link
ContributorAuthor

Seems that the CLA check has passed now.

AlexWaygood reacted with thumbs up emoji

Copy link
Member

@AlexWaygoodAlexWaygood left a comment•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks great to me. I verified the speedup by running this benchmark locally (which is a little different to the one you posted in your issue and the one you posted in this PR).

Benchmark:
importdataclassesimportstringimporttimeFoo=dataclasses.make_dataclass("Foo",    [(letter,int)forletterinstring.ascii_lowercase],frozen=True)classBar(Foo): ...instance=Bar(*range(26))t0=time.perf_counter()for_inrange(10_000_000):instance.foo=1delinstance.fooprint(f"{time.perf_counter()-t0:.2f}")

The result of this benchmark is 15.15 seconds onmain on my machine (--pgo non-debug build), but 6.44 seconds with this patch applied.

I'll wait for a thumbs-up from@ericvsmith,@carljm, or another core dev before merging, but this has my approval -- thanks!

XuehaiPan reacted with heart emoji
@AlexWaygoodAlexWaygood changed the titlegh-102578: Useset-based name lookup rather thantuples for frozen dataclassesgh-102578: Optimise setting and deleting mutable attributes on non-dataclass subclasses of frozen dataclassesMar 10, 2023
else:
# Special case for the zero-length tuple.
# Special case for the zero-length set.
# Use the empty tuple singleton to avoid unnecessary `set` construction
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Not that it matters much, but the zero length case could just avoid theor name in ... test entirely. Maybe fields_str should become fields_test, and then set it toor name in {<<generated set literal>>} or set it to an empty string if there are no fields. Then change the generated code tof'if type(self) is cls {fields_test}:' Although that doesn't read very well. Maybe tweak fields_test to be something else.

This could be part of a different PR, or include it here. But in any event I'm not positive that the zero length case actually has a test. We should make sure it does for this PR.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

But in any event I'm not positive that the zero length case actually has a test. We should make sure it does for this PR.

Added a small test for empty frozen dataclass.

Copy link
Member

@carljmcarljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Code changes LGTM. A couple comments on the new test.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be put in the comfy chair!

@XuehaiPan
Copy link
ContributorAuthor

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@carljm,@AlexWaygood: please review the changes made to this pull request.

Copy link
Member

@carljmcarljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks good to me; thanks for the perf improvement!

Copy link
Member

@AlexWaygoodAlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks great!

@ericvsmithericvsmith merged commitee6f841 intopython:mainMar 11, 2023
@XuehaiPanXuehaiPan deleted the dataclasses-lookup branchMarch 11, 2023 04:44
iritkatriel pushed a commit to iritkatriel/cpython that referenced this pull requestMar 12, 2023
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@ericvsmithericvsmithericvsmith left review comments

@carljmcarljmcarljm approved these changes

@AlexWaygoodAlexWaygoodAlexWaygood approved these changes

+1 more reviewer

@TeamSpen210TeamSpen210TeamSpen210 left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

performancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directory

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Enhancement] Speed up setting and deleting mutable attributes on non-dataclass subclasses of frozen dataclasses

6 participants

@XuehaiPan@bedevere-bot@AlexWaygood@carljm@ericvsmith@TeamSpen210
[8]ページ先頭
©2009-2025 Movatter.jp