Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
Open
Assignees
Description
Feature or enhancement
I've noticed this comment yesterday:
Lines 1128 to 1131 in1fd603f
| # _dataclass_getstate and _dataclass_setstate are needed for pickling frozen | |
| # classes with slots. These could be slightly more performant if we generated | |
| # the code instead of iterating over fields. But that can be a project for | |
| # another day, if performance becomes an issue. |
So, out of curiosity I've decided to try this out. How fast can I make it?
The results are here:
Testing ZeroFieldsDump time (before): 0.229174 secDump time (after) : 0.147407 secTesting OneFieldDump time (before): 0.260065 secDump time (after) : 0.197960 secTesting EightFieldsDump time (before): 0.331278 secDump time (after) : 0.192171 secTesting NestedFieldsDump time (before): 0.846335 secDump time (after) : 0.470739 secHere's the simple benchmark that I was using:
importpicklefromdataclassesimportdataclassfromtimeitimporttimeitdefperf(e):print('Testing',e.__class__.__name__)d=pickle.dumps(e)iterations=10000t1=timeit(lambda:pickle.dumps(e),number=iterations)t2=timeit(lambda:pickle.loads(d),number=iterations)print(f'Dump time:{t1:.6f} sec')print(f'Load time:{t2:.6f} sec')print()@dataclass(frozen=True,slots=True)classZeroFields:passperf(ZeroFields())@dataclass(frozen=True,slots=True)classOneField:foo:strperf(OneField('l'))@dataclass(frozen=True,slots=True)classEightFields:foo:strbar:intbaz:intspam:list[int]eggs:dict[str,str]x:booly:boolz:boolperf(EightFields("a",1,2, [1,2,3,4,5], {'a':'a','b':'b','c':'c'},True,False,True,))@dataclass(frozen=True,slots=True)classNestedFields:z1:ZeroFieldsz2:ZeroFieldse1:EightFieldse2:EightFieldse=EightFields("a",1,2, [1,2,3,4,5], {'a':'a','b':'b','c':'c'},True,False,True,)perf(NestedFields(ZeroFields(),ZeroFields(),e,e))
Here's the very rough version of what I am planning to do:
cls.__getstate__=_create_fn('__getstate__', ['self'], [f"return ({', '.join(f'self.{f.name}'forfinfields(cls))})"], )
Things to do:
- Refactor current example code to be inline with other code generators
- Add
__setstate__similar support
Does it look like a good enough speed up to make this change?
CC@ericvsmith and@carljm