Simple example: A descriptor that returns a constant¶

TheTen class is a descriptor whose__get__() method alwaysreturns the constant10:

classTen:def__get__(self,obj,objtype=None):return10

To use the descriptor, it must be stored as a class variable in another class:

classA:x=5# Regular class attributey=Ten()# Descriptor instance

An interactive session shows the difference between normal attribute lookupand descriptor lookup:

>>>a=A()# Make an instance of class A>>>a.x# Normal attribute lookup5>>>a.y# Descriptor lookup10

In thea.x attribute lookup, the dot operator finds'x':5in the class dictionary. In thea.y lookup, the dot operatorfinds a descriptor instance, recognized by its__get__ method.Calling that method returns10.

Note that the value10 is not stored in either the class dictionary or theinstance dictionary. Instead, the value10 is computed on demand.

This example shows how a simple descriptor works, but it isn’t very useful.For retrieving constants, normal attribute lookup would be better.

In the next section, we’ll create something more useful, a dynamic lookup.

Dynamic lookups¶

Interesting descriptors typically run computations instead of returningconstants:

importosclassDirectorySize:def__get__(self,obj,objtype=None):returnlen(os.listdir(obj.dirname))classDirectory:size=DirectorySize()# Descriptor instancedef__init__(self,dirname):self.dirname=dirname# Regular instance attribute

An interactive session shows that the lookup is dynamic — it computesdifferent, updated answers each time:

>>>s=Directory('songs')>>>g=Directory('games')>>>s.size# The songs directory has twenty files20>>>g.size# The games directory has three files3>>>os.remove('games/chess')# Delete a game>>>g.size# File count is automatically updated2

Besides showing how descriptors can run computations, this example alsoreveals the purpose of the parameters to__get__(). Theselfparameter issize, an instance ofDirectorySize. Theobj parameter iseitherg ors, an instance ofDirectory. It is theobj parameter thatlets the__get__() method learn the target directory. Theobjtypeparameter is the classDirectory.

Managed attributes¶

A popular use for descriptors is managing access to instance data. Thedescriptor is assigned to a public attribute in the class dictionary while theactual data is stored as a private attribute in the instance dictionary. Thedescriptor’s__get__() and__set__() methods are triggered whenthe public attribute is accessed.

In the following example,age is the public attribute and_age is theprivate attribute. When the public attribute is accessed, the descriptor logsthe lookup or update:

importlogginglogging.basicConfig(level=logging.INFO)classLoggedAgeAccess:def__get__(self,obj,objtype=None):value=obj._agelogging.info('Accessing%r giving%r','age',value)returnvaluedef__set__(self,obj,value):logging.info('Updating%r to%r','age',value)obj._age=valueclassPerson:age=LoggedAgeAccess()# Descriptor instancedef__init__(self,name,age):self.name=name# Regular instance attributeself.age=age# Calls __set__()defbirthday(self):self.age+=1# Calls both __get__() and __set__()

An interactive session shows that all access to the managed attributeage islogged, but that the regular attributename is not logged:

>>>mary=Person('Mary M',30)# The initial age update is loggedINFO:root:Updating 'age' to 30>>>dave=Person('David D',40)INFO:root:Updating 'age' to 40>>>vars(mary)# The actual data is in a private attribute{'name': 'Mary M', '_age': 30}>>>vars(dave){'name': 'David D', '_age': 40}>>>mary.age# Access the data and log the lookupINFO:root:Accessing 'age' giving 3030>>>mary.birthday()# Updates are logged as wellINFO:root:Accessing 'age' giving 30INFO:root:Updating 'age' to 31>>>dave.name# Regular attribute lookup isn't logged'David D'>>>dave.age# Only the managed attribute is loggedINFO:root:Accessing 'age' giving 4040

One major issue with this example is that the private name_age is hardwired intheLoggedAgeAccess class. That means that each instance can only have onelogged attribute and that its name is unchangeable. In the next example,we’ll fix that problem.

Customized names¶

When a class uses descriptors, it can inform each descriptor about whichvariable name was used.

In this example, thePerson class has two descriptor instances,name andage. When thePerson class is defined, it makes acallback to__set_name__() inLoggedAccess so that the field names canbe recorded, giving each descriptor its ownpublic_name andprivate_name:

importlogginglogging.basicConfig(level=logging.INFO)classLoggedAccess:def__set_name__(self,owner,name):self.public_name=nameself.private_name='_'+namedef__get__(self,obj,objtype=None):value=getattr(obj,self.private_name)logging.info('Accessing%r giving%r',self.public_name,value)returnvaluedef__set__(self,obj,value):logging.info('Updating%r to%r',self.public_name,value)setattr(obj,self.private_name,value)classPerson:name=LoggedAccess()# First descriptor instanceage=LoggedAccess()# Second descriptor instancedef__init__(self,name,age):self.name=name# Calls the first descriptorself.age=age# Calls the second descriptordefbirthday(self):self.age+=1

An interactive session shows that thePerson class has called__set_name__() so that the field names would be recorded. Herewe callvars() to look up the descriptor without triggering it:

>>>vars(vars(Person)['name']){'public_name': 'name', 'private_name': '_name'}>>>vars(vars(Person)['age']){'public_name': 'age', 'private_name': '_age'}

The new class now logs access to bothname andage:

>>>pete=Person('Peter P',10)INFO:root:Updating 'name' to 'Peter P'INFO:root:Updating 'age' to 10>>>kate=Person('Catherine C',20)INFO:root:Updating 'name' to 'Catherine C'INFO:root:Updating 'age' to 20

The twoPerson instances contain only the private names:

>>>vars(pete){'_name': 'Peter P', '_age': 10}>>>vars(kate){'_name': 'Catherine C', '_age': 20}

Closing thoughts¶

Adescriptor is what we call any object that defines__get__(),__set__(), or__delete__().

Optionally, descriptors can have a__set_name__() method. This is onlyused in cases where a descriptor needs to know either the class where it wascreated or the name of class variable it was assigned to. (This method, ifpresent, is called even if the class is not a descriptor.)

Descriptors get invoked by the dot operator during attribute lookup. If adescriptor is accessed indirectly withvars(some_class)[descriptor_name],the descriptor instance is returned without invoking it.

Descriptors only work when used as class variables. When put in instances,they have no effect.

The main motivation for descriptors is to provide a hook allowing objectsstored in class variables to control what happens during attribute lookup.

Traditionally, the calling class controls what happens during lookup.Descriptors invert that relationship and allow the data being looked-up tohave a say in the matter.

Descriptors are used throughout the language. It is how functions turn intobound methods. Common tools likeclassmethod(),staticmethod(),property(), andfunctools.cached_property() are all implemented asdescriptors.

Validator class¶

A validator is a descriptor for managed attribute access. Prior to storingany data, it verifies that the new value meets various type and rangerestrictions. If those restrictions aren’t met, it raises an exception toprevent data corruption at its source.

ThisValidator class is both anabstract base class and amanaged attribute descriptor:

fromabcimportABC,abstractmethodclassValidator(ABC):def__set_name__(self,owner,name):self.private_name='_'+namedef__get__(self,obj,objtype=None):returngetattr(obj,self.private_name)def__set__(self,obj,value):self.validate(value)setattr(obj,self.private_name,value)@abstractmethoddefvalidate(self,value):pass

Custom validators need to inherit fromValidator and must supply avalidate() method to test various restrictions as needed.

Custom validators¶

Here are three practical data validation utilities:

1. OneOf verifies that a value is one of a restricted set of options.

2. Number verifies that a value is either anint orfloat. Optionally, it verifies that a value is between a givenminimum or maximum.

3. String verifies that a value is astr. Optionally, itvalidates a given minimum or maximum length. It can validate auser-definedpredicate as well.

classOneOf(Validator):def__init__(self,*options):self.options=set(options)defvalidate(self,value):ifvaluenotinself.options:raiseValueError(f'Expected{value!r} to be one of{self.options!r}')classNumber(Validator):def__init__(self,minvalue=None,maxvalue=None):self.minvalue=minvalueself.maxvalue=maxvaluedefvalidate(self,value):ifnotisinstance(value,(int,float)):raiseTypeError(f'Expected{value!r} to be an int or float')ifself.minvalueisnotNoneandvalue<self.minvalue:raiseValueError(f'Expected{value!r} to be at least{self.minvalue!r}')ifself.maxvalueisnotNoneandvalue>self.maxvalue:raiseValueError(f'Expected{value!r} to be no more than{self.maxvalue!r}')classString(Validator):def__init__(self,minsize=None,maxsize=None,predicate=None):self.minsize=minsizeself.maxsize=maxsizeself.predicate=predicatedefvalidate(self,value):ifnotisinstance(value,str):raiseTypeError(f'Expected{value!r} to be an str')ifself.minsizeisnotNoneandlen(value)<self.minsize:raiseValueError(f'Expected{value!r} to be no smaller than{self.minsize!r}')ifself.maxsizeisnotNoneandlen(value)>self.maxsize:raiseValueError(f'Expected{value!r} to be no bigger than{self.maxsize!r}')ifself.predicateisnotNoneandnotself.predicate(value):raiseValueError(f'Expected{self.predicate} to be true for{value!r}')

Practical application¶

Here’s how the data validators can be used in a real class:

classComponent:name=String(minsize=3,maxsize=10,predicate=str.isupper)kind=OneOf('wood','metal','plastic')quantity=Number(minvalue=0)def__init__(self,name,kind,quantity):self.name=nameself.kind=kindself.quantity=quantity

The descriptors prevent invalid instances from being created:

>>>Component('Widget','metal',5)# Blocked: 'Widget' is not all uppercaseTraceback (most recent call last):...ValueError:Expected <method 'isupper' of 'str' objects> to be true for 'Widget'>>>Component('WIDGET','metle',5)# Blocked: 'metle' is misspelledTraceback (most recent call last):...ValueError:Expected 'metle' to be one of {'metal', 'plastic', 'wood'}>>>Component('WIDGET','metal',-5)# Blocked: -5 is negativeTraceback (most recent call last):...ValueError:Expected -5 to be at least 0>>>Component('WIDGET','metal','V')# Blocked: 'V' isn't a numberTraceback (most recent call last):...TypeError:Expected 'V' to be an int or float>>>c=Component('WIDGET','metal',5)# Allowed:  The inputs are valid

Abstract¶

Defines descriptors, summarizes the protocol, and shows how descriptors arecalled. Provides an example showing how object relational mappings work.

Learning about descriptors not only provides access to a larger toolset, itcreates a deeper understanding of how Python works.

Definition and introduction¶

In general, a descriptor is an attribute value that has one of the methods inthe descriptor protocol. Those methods are__get__(),__set__(),and__delete__(). If any of those methods are defined for anattribute, it is said to be adescriptor.

The default behavior for attribute access is to get, set, or delete theattribute from an object’s dictionary. For instance,a.x has a lookup chainstarting witha.__dict__['x'], thentype(a).__dict__['x'], andcontinuing through the method resolution order oftype(a). If thelooked-up value is an object defining one of the descriptor methods, then Pythonmay override the default behavior and invoke the descriptor method instead.Where this occurs in the precedence chain depends on which descriptor methodswere defined.

Descriptors are a powerful, general purpose protocol. They are the mechanismbehind properties, methods, static methods, class methods, andsuper(). They are used throughout Python itself. Descriptorssimplify the underlying C code and offer a flexible set of new tools foreveryday Python programs.

Descriptor protocol¶

descr.__get__(self,obj,type=None)

descr.__set__(self,obj,value)

descr.__delete__(self,obj)

That is all there is to it. Define any of these methods and an object isconsidered a descriptor and can override default behavior upon being looked upas an attribute.

If an object defines__set__() or__delete__(), it is considereda data descriptor. Descriptors that only define__get__() are callednon-data descriptors (they are often used for methods but other uses arepossible).

Data and non-data descriptors differ in how overrides are calculated withrespect to entries in an instance’s dictionary. If an instance’s dictionaryhas an entry with the same name as a data descriptor, the data descriptortakes precedence. If an instance’s dictionary has an entry with the samename as a non-data descriptor, the dictionary entry takes precedence.

To make a read-only data descriptor, define both__get__() and__set__() with the__set__() raising anAttributeError whencalled. Defining the__set__() method with an exception raisingplaceholder is enough to make it a data descriptor.

Overview of descriptor invocation¶

A descriptor can be called directly withdesc.__get__(obj) ordesc.__get__(None,cls).

But it is more common for a descriptor to be invoked automatically fromattribute access.

The expressionobj.x looks up the attributex in the chain ofnamespaces forobj. If the search finds a descriptor outside of theinstance__dict__, its__get__() method isinvoked according to the precedence rules listed below.

The details of invocation depend on whetherobj is an object, class, orinstance of super.

Invocation from an instance¶

Instance lookup scans through a chain of namespaces giving data descriptorsthe highest priority, followed by instance variables, then non-datadescriptors, then class variables, and lastly__getattr__() if it isprovided.

If a descriptor is found fora.x, then it is invoked with:desc.__get__(a,type(a)).

The logic for a dotted lookup is inobject.__getattribute__(). Here isa pure Python equivalent:

deffind_name_in_mro(cls,name,default):"Emulate _PyType_Lookup() in Objects/typeobject.c"forbaseincls.__mro__:ifnameinvars(base):returnvars(base)[name]returndefaultdefobject_getattribute(obj,name):"Emulate PyObject_GenericGetAttr() in Objects/object.c"null=object()objtype=type(obj)cls_var=find_name_in_mro(objtype,name,null)descr_get=getattr(type(cls_var),'__get__',null)ifdescr_getisnotnull:if(hasattr(type(cls_var),'__set__')orhasattr(type(cls_var),'__delete__')):returndescr_get(cls_var,obj,objtype)# data descriptorifhasattr(obj,'__dict__')andnameinvars(obj):returnvars(obj)[name]# instance variableifdescr_getisnotnull:returndescr_get(cls_var,obj,objtype)# non-data descriptorifcls_varisnotnull:returncls_var# class variableraiseAttributeError(name)

Note, there is no__getattr__() hook in the__getattribute__()code. That is why calling__getattribute__() directly or withsuper().__getattribute__ will bypass__getattr__() entirely.

Instead, it is the dot operator and thegetattr() function that areresponsible for invoking__getattr__() whenever__getattribute__()raises anAttributeError. Their logic is encapsulated in a helperfunction:

defgetattr_hook(obj,name):"Emulate slot_tp_getattr_hook() in Objects/typeobject.c"try:returnobj.__getattribute__(name)exceptAttributeError:ifnothasattr(type(obj),'__getattr__'):raisereturntype(obj).__getattr__(obj,name)# __getattr__

Invocation from a class¶

The logic for a dotted lookup such asA.x is intype.__getattribute__(). The steps are similar to those forobject.__getattribute__() but the instance dictionary lookup is replacedby a search through the class’smethod resolution order.

If a descriptor is found, it is invoked withdesc.__get__(None,A).

The full C implementation can be found intype_getattro() and_PyType_Lookup() inObjects/typeobject.c.

Invocation from super¶

The logic for super’s dotted lookup is in the__getattribute__() method forobject returned bysuper().

A dotted lookup such assuper(A,obj).m searchesobj.__class__.__mro__for the base classB immediately followingA and then returnsB.__dict__['m'].__get__(obj,A). If not a descriptor,m is returnedunchanged.

The full C implementation can be found insuper_getattro() inObjects/typeobject.c. A pure Python equivalent can be found inGuido’s Tutorial.

Summary of invocation logic¶

The mechanism for descriptors is embedded in the__getattribute__()methods forobject,type, andsuper().

The important points to remember are:

• Descriptors are invoked by the__getattribute__() method.

• Classes inherit this machinery fromobject,type, orsuper().

• Overriding__getattribute__() prevents automatic descriptor callsbecause all the descriptor logic is in that method.

• object.__getattribute__() andtype.__getattribute__() makedifferent calls to__get__(). The first includes the instance and mayinclude the class. The second puts inNone for the instance and alwaysincludes the class.

• Data descriptors always override instance dictionaries.

• Non-data descriptors may be overridden by instance dictionaries.

Automatic name notification¶

Sometimes it is desirable for a descriptor to know what class variable name itwas assigned to. When a new class is created, thetype metaclassscans the dictionary of the new class. If any of the entries are descriptorsand if they define__set_name__(), that method is called with twoarguments. Theowner is the class where the descriptor is used, and thename is the class variable the descriptor was assigned to.

The implementation details are intype_new() andset_names() inObjects/typeobject.c.

Since the update logic is intype.__new__(), notifications only takeplace at the time of class creation. If descriptors are added to the classafterwards,__set_name__() will need to be called manually.

ORM example¶

The following code is a simplified skeleton showing how data descriptors couldbe used to implement anobject relational mapping.

The essential idea is that the data is stored in an external database. ThePython instances only hold keys to the database’s tables. Descriptors takecare of lookups or updates:

classField:def__set_name__(self,owner,name):self.fetch=f'SELECT{name} FROM{owner.table} WHERE{owner.key}=?;'self.store=f'UPDATE{owner.table} SET{name}=? WHERE{owner.key}=?;'def__get__(self,obj,objtype=None):returnconn.execute(self.fetch,[obj.key]).fetchone()[0]def__set__(self,obj,value):conn.execute(self.store,[value,obj.key])conn.commit()

We can use theField class to definemodels that describe the schema foreach table in a database:

classMovie:table='Movies'# Table namekey='title'# Primary keydirector=Field()year=Field()def__init__(self,key):self.key=keyclassSong:table='Music'key='title'artist=Field()year=Field()genre=Field()def__init__(self,key):self.key=key

To use the models, first connect to the database:

>>>importsqlite3>>>conn=sqlite3.connect('entertainment.db')

An interactive session shows how data is retrieved from the database and howit can be updated:

>>>Movie('Star Wars').director'George Lucas'>>>jaws=Movie('Jaws')>>>f'Released in{jaws.year} by{jaws.director}''Released in 1975 by Steven Spielberg'>>>Song('Country Roads').artist'John Denver'>>>Movie('Star Wars').director='J.J. Abrams'>>>Movie('Star Wars').director'J.J. Abrams'

Properties¶

Callingproperty() is a succinct way of building a data descriptor thattriggers a function call upon access to an attribute. Its signature is:

property(fget=None,fset=None,fdel=None,doc=None)->property

The documentation shows a typical use to define a managed attributex:

classC:defgetx(self):returnself.__xdefsetx(self,value):self.__x=valuedefdelx(self):delself.__xx=property(getx,setx,delx,"I'm the 'x' property.")

To see howproperty() is implemented in terms of the descriptor protocol,here is a pure Python equivalent that implements most of the core functionality:

classProperty:"Emulate PyProperty_Type() in Objects/descrobject.c"def__init__(self,fget=None,fset=None,fdel=None,doc=None):self.fget=fgetself.fset=fsetself.fdel=fdelifdocisNoneandfgetisnotNone:doc=fget.__doc__self.__doc__=docdef__set_name__(self,owner,name):self.__name__=namedef__get__(self,obj,objtype=None):ifobjisNone:returnselfifself.fgetisNone:raiseAttributeErrorreturnself.fget(obj)def__set__(self,obj,value):ifself.fsetisNone:raiseAttributeErrorself.fset(obj,value)def__delete__(self,obj):ifself.fdelisNone:raiseAttributeErrorself.fdel(obj)defgetter(self,fget):returntype(self)(fget,self.fset,self.fdel,self.__doc__)defsetter(self,fset):returntype(self)(self.fget,fset,self.fdel,self.__doc__)defdeleter(self,fdel):returntype(self)(self.fget,self.fset,fdel,self.__doc__)

Theproperty() builtin helps whenever a user interface has grantedattribute access and then subsequent changes require the intervention of amethod.

For instance, a spreadsheet class may grant access to a cell value throughCell('b10').value. Subsequent improvements to the program require the cellto be recalculated on every access; however, the programmer does not want toaffect existing client code accessing the attribute directly. The solution isto wrap access to the value attribute in a property data descriptor:

classCell:...@propertydefvalue(self):"Recalculate the cell before returning value"self.recalc()returnself._value

Either the built-inproperty() or ourProperty() equivalent wouldwork in this example.

Functions and methods¶

Python’s object oriented features are built upon a function based environment.Using non-data descriptors, the two are merged seamlessly.

Functions stored in class dictionaries get turned into methods when invoked.Methods only differ from regular functions in that the object instance isprepended to the other arguments. By convention, the instance is calledself but could be calledthis or any other variable name.

Methods can be created manually withtypes.MethodType which isroughly equivalent to:

classMethodType:"Emulate PyMethod_Type in Objects/classobject.c"def__init__(self,func,obj):self.__func__=funcself.__self__=objdef__call__(self,*args,**kwargs):func=self.__func__obj=self.__self__returnfunc(obj,*args,**kwargs)def__getattribute__(self,name):"Emulate method_getset() in Objects/classobject.c"ifname=='__doc__':returnself.__func__.__doc__returnobject.__getattribute__(self,name)def__getattr__(self,name):"Emulate method_getattro() in Objects/classobject.c"returngetattr(self.__func__,name)def__get__(self,obj,objtype=None):"Emulate method_descr_get() in Objects/classobject.c"returnself

To support automatic creation of methods, functions include the__get__() method for binding methods during attribute access. Thismeans that functions are non-data descriptors that return bound methodsduring dotted lookup from an instance. Here’s how it works:

classFunction:...def__get__(self,obj,objtype=None):"Simulate func_descr_get() in Objects/funcobject.c"ifobjisNone:returnselfreturnMethodType(self,obj)

Running the following class in the interpreter shows how the functiondescriptor works in practice:

classD:deff(self):returnselfclassD2:pass

The function has aqualified name attribute to support introspection:

>>>D.f.__qualname__'D.f'

Accessing the function through the class dictionary does not invoke__get__(). Instead, it just returns the underlying function object:

>>>D.__dict__['f']<function D.f at 0x00C45070>

Dotted access from a class calls__get__() which just returns theunderlying function unchanged:

>>>D.f<function D.f at 0x00C45070>

The interesting behavior occurs during dotted access from an instance. Thedotted lookup calls__get__() which returns a bound method object:

>>>d=D()>>>d.f<bound method D.f of <__main__.D object at 0x00B18C90>>

Internally, the bound method stores the underlying function and the boundinstance:

>>>d.f.__func__<function D.f at 0x00C45070>>>>d.f.__self__<__main__.D object at 0x00B18C90>

If you have ever wondered whereself comes from in regular methods or wherecls comes from in class methods, this is it!

Kinds of methods¶

Non-data descriptors provide a simple mechanism for variations on the usualpatterns of binding functions into methods.

To recap, functions have a__get__() method so that they can be convertedto a method when accessed as attributes. The non-data descriptor transforms anobj.f(*args) call intof(obj,*args). Callingcls.f(*args)becomesf(*args).

This chart summarizes the binding and its two most useful variants:

Transformation	Called from anobject	Called from aclass
function	f(obj, *args)	f(*args)
staticmethod	f(*args)	f(*args)
classmethod	f(type(obj), *args)	f(cls, *args)

Static methods¶

Static methods return the underlying function without changes. Calling eitherc.f orC.f is the equivalent of a direct lookup intoobject.__getattribute__(c,"f") orobject.__getattribute__(C,"f"). As aresult, the function becomes identically accessible from either an object or aclass.

Good candidates for static methods are methods that do not reference theself variable.

For instance, a statistics package may include a container class forexperimental data. The class provides normal methods for computing the average,mean, median, and other descriptive statistics that depend on the data. However,there may be useful functions which are conceptually related but do not dependon the data. For instance,erf(x) is handy conversion routine that comes upin statistical work but does not directly depend on a particular dataset.It can be called either from an object or the class:s.erf(1.5)-->0.9332orSample.erf(1.5)-->0.9332.

Since static methods return the underlying function with no changes, theexample calls are unexciting:

classE:@staticmethoddeff(x):returnx*10

>>>E.f(3)30>>>E().f(3)30

Using the non-data descriptor protocol, a pure Python version ofstaticmethod() would look like this:

importfunctoolsclassStaticMethod:"Emulate PyStaticMethod_Type() in Objects/funcobject.c"def__init__(self,f):self.f=ffunctools.update_wrapper(self,f)def__get__(self,obj,objtype=None):returnself.fdef__call__(self,*args,**kwds):returnself.f(*args,**kwds)

Thefunctools.update_wrapper() call adds a__wrapped__ attributethat refers to the underlying function. Also it carries forwardthe attributes necessary to make the wrapper look like the wrappedfunction:__name__,__qualname__,__doc__, and__annotations__.

Class methods¶

Unlike static methods, class methods prepend the class reference to theargument list before calling the function. This format is the samefor whether the caller is an object or a class:

classF:@classmethoddeff(cls,x):returncls.__name__,x

>>>F.f(3)('F', 3)>>>F().f(3)('F', 3)

This behavior is useful whenever the method only needs to have a classreference and does not rely on data stored in a specific instance. One use forclass methods is to create alternate class constructors. For example, theclassmethoddict.fromkeys() creates a new dictionary from a list ofkeys. The pure Python equivalent is:

classDict(dict):@classmethoddeffromkeys(cls,iterable,value=None):"Emulate dict_fromkeys() in Objects/dictobject.c"d=cls()forkeyiniterable:d[key]=valuereturnd

Now a new dictionary of unique keys can be constructed like this:

>>>d=Dict.fromkeys('abracadabra')>>>type(d)isDictTrue>>>d{'a': None, 'b': None, 'r': None, 'c': None, 'd': None}

Using the non-data descriptor protocol, a pure Python version ofclassmethod() would look like this:

importfunctoolsclassClassMethod:"Emulate PyClassMethod_Type() in Objects/funcobject.c"def__init__(self,f):self.f=ffunctools.update_wrapper(self,f)def__get__(self,obj,cls=None):ifclsisNone:cls=type(obj)returnMethodType(self.f,cls)

Thefunctools.update_wrapper() call inClassMethod adds a__wrapped__ attribute that refers to the underlying function. Alsoit carries forward the attributes necessary to make the wrapper looklike the wrapped function:__name__,__qualname__,__doc__,and__annotations__.

Member objects and __slots__¶

When a class defines__slots__, it replaces instance dictionaries with afixed-length array of slot values. From a user point of view that hasseveral effects:

1. Provides immediate detection of bugs due to misspelled attributeassignments. Only attribute names specified in__slots__ are allowed:

classVehicle:__slots__=('id_number','make','model')

>>>auto=Vehicle()>>>auto.id_nubmer='VYE483814LQEX'Traceback (most recent call last):...AttributeError:'Vehicle' object has no attribute 'id_nubmer'

2. Helps create immutable objects where descriptors manage access to privateattributes stored in__slots__:

classImmutable:__slots__=('_dept','_name')# Replace the instance dictionarydef__init__(self,dept,name):self._dept=dept# Store to private attributeself._name=name# Store to private attribute@property# Read-only descriptordefdept(self):returnself._dept@propertydefname(self):# Read-only descriptorreturnself._name

>>>mark=Immutable('Botany','Mark Watney')>>>mark.dept'Botany'>>>mark.dept='Space Pirate'Traceback (most recent call last):...AttributeError:property 'dept' of 'Immutable' object has no setter>>>mark.location='Mars'Traceback (most recent call last):...AttributeError:'Immutable' object has no attribute 'location'

3. Saves memory. On a 64-bit Linux build, an instance with two attributestakes 48 bytes with__slots__ and 152 bytes without. Thisflyweightdesign pattern likely onlymatters when a large number of instances are going to be created.

4. Improves speed. Reading instance variables is 35% faster with__slots__ (as measured with Python 3.10 on an Apple M1 processor).

5. Blocks tools likefunctools.cached_property() which require aninstance dictionary to function correctly:

fromfunctoolsimportcached_propertyclassCP:__slots__=()# Eliminates the instance dict@cached_property# Requires an instance dictdefpi(self):return4*sum((-1.0)**n/(2.0*n+1.0)forninreversed(range(100_000)))

>>>CP().piTraceback (most recent call last):...TypeError:No '__dict__' attribute on 'CP' instance to cache 'pi' property.

It is not possible to create an exact drop-in pure Python version of__slots__ because it requires direct access to C structures and controlover object memory allocation. However, we can build a mostly faithfulsimulation where the actual C structure for slots is emulated by a private_slotvalues list. Reads and writes to that private structure are managedby member descriptors:

null=object()classMember:def__init__(self,name,clsname,offset):'Emulate PyMemberDef in Include/structmember.h'# Also see descr_new() in Objects/descrobject.cself.name=nameself.clsname=clsnameself.offset=offsetdef__get__(self,obj,objtype=None):'Emulate member_get() in Objects/descrobject.c'# Also see PyMember_GetOne() in Python/structmember.cifobjisNone:returnselfvalue=obj._slotvalues[self.offset]ifvalueisnull:raiseAttributeError(self.name)returnvaluedef__set__(self,obj,value):'Emulate member_set() in Objects/descrobject.c'obj._slotvalues[self.offset]=valuedef__delete__(self,obj):'Emulate member_delete() in Objects/descrobject.c'value=obj._slotvalues[self.offset]ifvalueisnull:raiseAttributeError(self.name)obj._slotvalues[self.offset]=nulldef__repr__(self):'Emulate member_repr() in Objects/descrobject.c'returnf'<Member{self.name!r} of{self.clsname!r}>'

Thetype.__new__() method takes care of adding member objects to classvariables:

classType(type):'Simulate how the type metaclass adds member objects for slots'def__new__(mcls,clsname,bases,mapping,**kwargs):'Emulate type_new() in Objects/typeobject.c'# type_new() calls PyTypeReady() which calls add_methods()slot_names=mapping.get('slot_names',[])foroffset,nameinenumerate(slot_names):mapping[name]=Member(name,clsname,offset)returntype.__new__(mcls,clsname,bases,mapping,**kwargs)

Theobject.__new__() method takes care of creating instances that haveslots instead of an instance dictionary. Here is a rough simulation in purePython:

classObject:'Simulate how object.__new__() allocates memory for __slots__'def__new__(cls,*args,**kwargs):'Emulate object_new() in Objects/typeobject.c'inst=super().__new__(cls)ifhasattr(cls,'slot_names'):empty_slots=[null]*len(cls.slot_names)object.__setattr__(inst,'_slotvalues',empty_slots)returninstdef__setattr__(self,name,value):'Emulate _PyObject_GenericSetAttrWithDict() Objects/object.c'cls=type(self)ifhasattr(cls,'slot_names')andnamenotincls.slot_names:raiseAttributeError(f'{cls.__name__!r} object has no attribute{name!r}')super().__setattr__(name,value)def__delattr__(self,name):'Emulate _PyObject_GenericSetAttrWithDict() Objects/object.c'cls=type(self)ifhasattr(cls,'slot_names')andnamenotincls.slot_names:raiseAttributeError(f'{cls.__name__!r} object has no attribute{name!r}')super().__delattr__(name)

To use the simulation in a real class, just inherit fromObject andset themetaclass toType:

classH(Object,metaclass=Type):'Instance variables stored in slots'slot_names=['x','y']def__init__(self,x,y):self.x=xself.y=y

At this point, the metaclass has loaded member objects forx andy:

>>>frompprintimportpp>>>pp(dict(vars(H))){'__module__': '__main__', '__doc__': 'Instance variables stored in slots', 'slot_names': ['x', 'y'], '__init__': <function H.__init__ at 0x7fb5d302f9d0>, 'x': <Member 'x' of 'H'>, 'y': <Member 'y' of 'H'>}

When instances are created, they have aslot_values list where theattributes are stored:

>>>h=H(10,20)>>>vars(h){'_slotvalues': [10, 20]}>>>h.x=55>>>vars(h){'_slotvalues': [55, 20]}

Misspelled or unassigned attributes will raise an exception:

>>>h.xzTraceback (most recent call last):...AttributeError:'H' object has no attribute 'xz'