The reasons for the 2024 Steering Council rejection include:
PEP 557 addeddataclasses to the Python stdlib.PEP 681 addeddataclass_transform() to help type checkers understandseveral common dataclass-like libraries, such as attrs, Pydantic, and objectrelational mapper (ORM) packages such as SQLAlchemy and Django.
A common feature other libraries provide over the standard libraryimplementation is the ability for the library to convert arguments given atinitialization time into the types expected for each field using auser-provided conversion function.
Therefore, this PEP adds aconverter parameter todataclasses.field()(along with the requisite changes todataclasses.Field anddataclass_transform()) to specify the function to use toconvert the input value for each field to the representation to be stored inthe dataclass.
There is no existing, standard way fordataclasses or third-partydataclass-like libraries to support argument conversion in a type-checkableway. To work around this limitation, library authors/users are forced to chooseto:
__init__ which declares “wider” parameter types andconverts them when setting the appropriate attribute. This not only duplicatesthe typing annotations between the converter and__init__, but also optsthe user out of many of the featuresdataclasses provides.__init__ but without meaningful type annotationsfor the parameter types requiring conversion.None of these choices are ideal.
Adding argument conversion semantics is useful and beneficial enough that mostdataclass-like libraries provide support. Adding this feature to the standardlibrary means more users are able to opt-in to these benefits without requiringthird-party libraries. Additionally third-party libraries are able to cluetype-checkers into their own conversion semantics through added support indataclass_transform(), meaning users of those librariesbenefit as well.
converter parameterThis specification introduces a new parameter namedconverter to thedataclasses.field() function. If provided, it represents a single-argumentcallable used to convert all values when assigning to the associated attribute.
For frozen dataclasses, the converter is only used inside adataclass-synthesized__init__ when setting the attribute. For non-frozen dataclasses, the converteris used for all attribute assignment (E.g.obj.attr=value), which includesassignment of default values.
The converter is not used when reading attributes, as the attributes should alreadyhave been converted.
Adding this parameter also implies the following changes:
converter attribute will be added todataclasses.Field.converter will be added todataclass_transform()’slist of supported field specifier parameters.defstr_or_none(x:Any)->str|None:returnstr(x)ifxisnotNoneelseNone@dataclasses.dataclassclassInventoryItem:# `converter` as a type (including a GenericAlias).id:int=dataclasses.field(converter=int)skus:tuple[int,...]=dataclasses.field(converter=tuple[int,...])# `converter` as a callable.vendor:str|None=dataclasses.field(converter=str_or_none))names:tuple[str,...]=dataclasses.field(converter=lambdanames:tuple(map(str.lower,names)))# Note that lambdas are supported, but discouraged as they are untyped.# The default value is also converted; therefore the following is not a# type error.stock_image_path:pathlib.PurePosixPath=dataclasses.field(converter=pathlib.PurePosixPath,default="assets/unknown.png")# Default value conversion extends to `default_factory`;# therefore the following is also not a type error.shelves:tuple=dataclasses.field(converter=tuple,default_factory=list)item1=InventoryItem("1",[234,765],None,["PYTHON PLUSHIE","FLUFFY SNAKE"])# item1's repr would be (with added newlines for readability):# InventoryItem(# id=1,# skus=(234, 765),# vendor=None,# names=('PYTHON PLUSHIE', 'FLUFFY SNAKE'),# stock_image_path=PurePosixPath('assets/unknown.png'),# shelves=()# )# Attribute assignment also participates in conversion.item1.skus=[555]# item1's skus attribute is now (555,).
Aconverter must be a callable that accepts a single positional argument, andthe parameter type corresponding to this positional argument provides the typeof the the synthesized__init__ parameter associated with the field.
In other words, the argument provided for the converter parameter must becompatible withCallable[[T],X] whereT is the input type forthe converter andX is the output type of the converter.
default anddefault_factoryBecause default values are unconditionally converted usingconverter, ifan argument forconverter is provided alongside eitherdefault ordefault_factory, the type of the default (thedefault argument ifprovided, otherwise the return value ofdefault_factory) should be checkedusing the type of the single argument to theconverter callable.
The return type of the callable must be a type that’s compatible with thefield’s declared type. This includes the field’s type exactly, but can also bea type that’s more specialized (such as a converter returning alist[int]for a field annotated aslist, or a converter returning anint for afield annotated asint|str).
One downside introduced by this PEP is that knowing what argument types areallowed in the dataclass’__init__ and during attribute assignment is notimmediately obvious from reading the dataclass. The allowable types are definedby the converter.
This is true when reading code from source, however typing-related aides suchastyping.reveal_type and “IntelliSense” in an IDE should make it easy to knowexactly what types are allowed without having to read any source code.
These changes don’t introduce any compatibility problems since theyonly introduce opt-in new features.
There are no direct security concerns with these changes.
Documentation and examples explaining the new parameter and behavior will beadded to the relevant sections of the docs site (primarily ondataclasses) and linked from theWhat’s New document.
The added documentation/examples will also cover the “common pitfalls” thatusers of converters are likely to encounter. Such pitfalls include:
None/sentinel values.__init__parameter’s type will becomeAny.__init__ infrozen dataclasses.__setattr__ innon-frozen dataclasses.Additionally, potentially confusing pattern matching semantics should be covered:
@dataclassclassPoint:x:int=field(converter=int)y:intmatchPoint(x="0",y=0):casePoint(x="0",y=0):# Won't be matched...casePoint():# Will be matched...case_:...
However it’s worth noting this behavior is true of any type that does conversionin its initializer, and type-checkers should be able to catch this pitfall:
matchint("0"):caseint("0"):# Won't be matched...case_:# Will be matched...
The attrs libraryalready includes aconverterparameter exhibiting the same converter semantics (converting in theinitializer and on attribute setting) when using the@define classdecorator.
CPython support is implemented ona branch in the author’s fork.
typing.dataclass_transform’sfield_specifiersThe idea of isolating this addition todataclass_transform() was brieflydiscussed on Typing-SIG where it was suggestedto broaden this todataclasses more generally.
Additionally, adding this todataclasses ensures anyone can reap thebenefits without requiring additional libraries.
There are pros and cons with both converting and not converting default values.Leaving default values as-is allows type-checkers and dataclass authors toexpect that the type of the default matches the type of the field. However,converting default values has three large advantages:
One idea could be to allow the type of the field specified (e.g.str orint) to be used as a converter for each argument provided.Pydantic’s data conversion has semantics whichappear to be similar to this approach.
This works well for fairly simple types, but leads to ambiguity in expectedbehavior for complex types such as generics. E.g. Fortuple[int,...] it isambiguous if the converter is supposed to simply convert an iterable to a tuple,or if it is additionally supposed to convert each element type toint. Orforint|None, which isn’t callable.
Another idea would be to allow the user to omit the attribute’s type annotationif providing afield with aconverter argument. Although this wouldreduce the common repetition this PEP introduces (e.g.x:str=field(converter=str)),it isn’t clear how to best support this while maintaining the current dataclasssemantics (namely, that the attribute order is preserved for things like thesynthesized__init__, ordataclasses.fields). This is because there isn’tan easy way in Python (today) to get the annotation-only attributes interspersedwith un-annotated attributes in the order they were defined.
A sentinel annotation could be applied (e.g.x:FromConverter=...),however this breaks a fundamental assumption of type annotations.
Lastly, this is feasible ifall fields (including those without a converter)were assigned todataclasses.field, which means the class’ own namespaceholds the order, however this trades repetition of type+converter withrepetition of field assignment. The end result is no gain or loss of repetition,but with the added complexity of dataclasses semantics.
This PEP doesn’t suggest it can’t or shouldn’t be done. Just that it isn’tincluded in this PEP.
This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0712.rst
Last modified:2025-02-01 08:55:40 GMT