functools — Higher-order functions and operations on callable objects¶

Source code:Lib/functools.py

Thefunctools module is for higher-order functions: functions that act onor return other functions. In general, any callable object can be treated as afunction for the purposes of this module.

Thefunctools module defines the following functions:

@functools.cache(user_function)¶

Simple lightweight unbounded function cache. Sometimes called“memoize”.

Returns the same aslru_cache(maxsize=None), creating a thinwrapper around a dictionary lookup for the function arguments. Because itnever needs to evict old values, this is smaller and faster thanlru_cache() with a size limit.

For example:

@cachedeffactorial(n):returnn*factorial(n-1)ifnelse1>>>factorial(10)# no previously cached result, makes 11 recursive calls3628800>>>factorial(5)# just looks up cached value result120>>>factorial(12)# makes two new recursive calls, the other 10 are cached479001600

The cache is threadsafe so that the wrapped function can be used inmultiple threads. This means that the underlying data structure willremain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once ifanother thread makes an additional call before the initial call has beencompleted and cached.

Added in version 3.9.

@functools.cached_property(func)¶

Transform a method of a class into a property whose value is computed onceand then cached as a normal attribute for the life of the instance. Similartoproperty(), with the addition of caching. Useful for expensivecomputed properties of instances that are otherwise effectively immutable.

Example:

classDataSet:def__init__(self,sequence_of_numbers):self._data=tuple(sequence_of_numbers)@cached_propertydefstdev(self):returnstatistics.stdev(self._data)

The mechanics ofcached_property() are somewhat different fromproperty(). A regular property blocks attribute writes unless asetter is defined. In contrast, acached_property allows writes.

Thecached_property decorator only runs on lookups and only when anattribute of the same name doesn’t exist. When it does run, thecached_property writes to the attribute with the same name. Subsequentattribute reads and writes take precedence over thecached_propertymethod and it works like a normal attribute.

The cached value can be cleared by deleting the attribute. Thisallows thecached_property method to run again.

Thecached_property does not prevent a possible race condition inmulti-threaded usage. The getter function could run more than once on thesame instance, with the latest run setting the cached value. If the cachedproperty is idempotent or otherwise not harmful to run more than once on aninstance, this is fine. If synchronization is needed, implement the necessarylocking inside the decorated getter function or around the cached propertyaccess.

Note, this decorator interferes with the operation ofPEP 412key-sharing dictionaries. This means that instance dictionariescan take more space than usual.

Also, this decorator requires that the__dict__ attribute on each instancebe a mutable mapping. This means it will not work with some types, such asmetaclasses (since the__dict__ attributes on type instances areread-only proxies for the class namespace), and those that specify__slots__ without including__dict__ as one of the defined slots(as such classes don’t provide a__dict__ attribute at all).

If a mutable mapping is not available or if space-efficient key sharing isdesired, an effect similar tocached_property() can also be achieved bystackingproperty() on top oflru_cache(). SeeHow do I cache method calls? for more details on how this differs fromcached_property().

Added in version 3.8.

Changed in version 3.12:Prior to Python 3.12,cached_property included an undocumented lock toensure that in multi-threaded usage the getter function was guaranteed torun only once per instance. However, the lock was per-property, notper-instance, which could result in unacceptably high lock contention. InPython 3.12+ this locking is removed.

functools.cmp_to_key(func)¶

Transform an old-style comparison function to akey function. Usedwith tools that accept key functions (such assorted(),min(),max(),heapq.nlargest(),heapq.nsmallest(),itertools.groupby()). This function is primarily used as a transitiontool for programs being converted from Python 2 which supported the use ofcomparison functions.

A comparison function is any callable that accepts two arguments, compares them,and returns a negative number for less-than, zero for equality, or a positivenumber for greater-than. A key function is a callable that accepts oneargument and returns another value to be used as the sort key.

Example:

sorted(iterable,key=cmp_to_key(locale.strcoll))# locale-aware sort order

For sorting examples and a brief sorting tutorial, seeSorting Techniques.

Added in version 3.2.

@functools.lru_cache(user_function)¶

@functools.lru_cache(maxsize=128,typed=False)

Decorator to wrap a function with a memoizing callable that saves up to themaxsize most recent calls. It can save time when an expensive or I/O boundfunction is periodically called with the same arguments.

The cache is threadsafe so that the wrapped function can be used inmultiple threads. This means that the underlying data structure willremain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once ifanother thread makes an additional call before the initial call has beencompleted and cached.

Since a dictionary is used to cache results, the positional and keywordarguments to the function must behashable.

Distinct argument patterns may be considered to be distinct calls withseparate cache entries. For example,f(a=1,b=2) andf(b=2,a=1)differ in their keyword argument order and may have two separate cacheentries.

Ifuser_function is specified, it must be a callable. This allows thelru_cache decorator to be applied directly to a user function, leavingthemaxsize at its default value of 128:

@lru_cachedefcount_vowels(sentence):returnsum(sentence.count(vowel)forvowelin'AEIOUaeiou')

Ifmaxsize is set toNone, the LRU feature is disabled and the cache cangrow without bound.

Iftyped is set to true, function arguments of different types will becached separately. Iftyped is false, the implementation will usuallyregard them as equivalent calls and only cache a single result. (Sometypes such asstr andint may be cached separately even whentypedis false.)

Note, type specificity applies only to the function’s immediate argumentsrather than their contents. The scalar arguments,Decimal(42) andFraction(42) are be treated as distinct calls with distinct results.In contrast, the tuple arguments('answer',Decimal(42)) and('answer',Fraction(42)) are treated as equivalent.

The wrapped function is instrumented with acache_parameters()function that returns a newdict showing the values formaxsizeandtyped. This is for information purposes only. Mutating the valueshas no effect.

To help measure the effectiveness of the cache and tune themaxsizeparameter, the wrapped function is instrumented with acache_info()function that returns anamed tuple showinghits,misses,maxsize andcurrsize.

The decorator also provides acache_clear() function for clearing orinvalidating the cache.

The original underlying function is accessible through the__wrapped__ attribute. This is useful for introspection, forbypassing the cache, or for rewrapping the function with a different cache.

The cache keeps references to the arguments and return values until they ageout of the cache or until the cache is cleared.

If a method is cached, theself instance argument is included in thecache. SeeHow do I cache method calls?

AnLRU (least recently used) cacheworks best when the most recent calls are the best predictors of upcomingcalls (for example, the most popular articles on a news server tend tochange each day). The cache’s size limit assures that the cache does notgrow without bound on long-running processes such as web servers.

In general, the LRU cache should only be used when you want to reusepreviously computed values. Accordingly, it doesn’t make sense to cachefunctions with side-effects, functions that need to createdistinct mutable objects on each call (such as generators and async functions),or impure functions such as time() or random().

Example of an LRU cache for static web content:

@lru_cache(maxsize=32)defget_pep(num):'Retrieve text of a Python Enhancement Proposal'resource=f'https://peps.python.org/pep-{num:04d}'try:withurllib.request.urlopen(resource)ass:returns.read()excepturllib.error.HTTPError:return'Not Found'>>>fornin8,290,308,320,8,218,320,279,289,320,9991:...pep=get_pep(n)...print(n,len(pep))>>>get_pep.cache_info()CacheInfo(hits=3,misses=8,maxsize=32,currsize=8)

Example of efficiently computingFibonacci numbersusing a cache to implement adynamic programmingtechnique:

@lru_cache(maxsize=None)deffib(n):ifn<2:returnnreturnfib(n-1)+fib(n-2)>>>[fib(n)forninrange(16)][0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,610]>>>fib.cache_info()CacheInfo(hits=28,misses=16,maxsize=None,currsize=16)

Added in version 3.2.

Changed in version 3.3:Added thetyped option.

Changed in version 3.8:Added theuser_function option.

Changed in version 3.9:Added the functioncache_parameters()

@functools.total_ordering¶

Given a class defining one or more rich comparison ordering methods, thisclass decorator supplies the rest. This simplifies the effort involvedin specifying all of the possible rich comparison operations:

The class must define one of__lt__(),__le__(),__gt__(), or__ge__().In addition, the class should supply an__eq__() method.

For example:

@total_orderingclassStudent:def_is_valid_operand(self,other):return(hasattr(other,"lastname")andhasattr(other,"firstname"))def__eq__(self,other):ifnotself._is_valid_operand(other):returnNotImplementedreturn((self.lastname.lower(),self.firstname.lower())==(other.lastname.lower(),other.firstname.lower()))def__lt__(self,other):ifnotself._is_valid_operand(other):returnNotImplementedreturn((self.lastname.lower(),self.firstname.lower())<(other.lastname.lower(),other.firstname.lower()))

Note

While this decorator makes it easy to create well behaved totallyordered types, itdoes come at the cost of slower execution andmore complex stack traces for the derived comparison methods. Ifperformance benchmarking indicates this is a bottleneck for a givenapplication, implementing all six rich comparison methods instead islikely to provide an easy speed boost.

Note

This decorator makes no attempt to override methods that have beendeclared in the classor its superclasses. Meaning that if asuperclass defines a comparison operator,total_ordering will notimplement it again, even if the original method is abstract.

Added in version 3.2.

Changed in version 3.4:ReturningNotImplemented from the underlying comparison function forunrecognised types is now supported.

functools.partial(func,/,*args,**keywords)¶

Return a newpartial object which when calledwill behave likefunc called with the positional argumentsargsand keyword argumentskeywords. If more arguments are supplied to thecall, they are appended toargs. If additional keyword arguments aresupplied, they extend and overridekeywords.Roughly equivalent to:

defpartial(func,/,*args,**keywords):defnewfunc(*fargs,**fkeywords):newkeywords={**keywords,**fkeywords}returnfunc(*args,*fargs,**newkeywords)newfunc.func=funcnewfunc.args=argsnewfunc.keywords=keywordsreturnnewfunc

Thepartial() is used for partial function application which “freezes”some portion of a function’s arguments and/or keywords resulting in a new objectwith a simplified signature. For example,partial() can be used to createa callable that behaves like theint() function where thebase argumentdefaults to two:

>>>fromfunctoolsimportpartial>>>basetwo=partial(int,base=2)>>>basetwo.__doc__='Convert base 2 string to an int.'>>>basetwo('10010')18

classfunctools.partialmethod(func,/,*args,**keywords)¶

Return a newpartialmethod descriptor which behaveslikepartial except that it is designed to be used as a methoddefinition rather than being directly callable.

func must be adescriptor or a callable (objects which are both,like normal functions, are handled as descriptors).

Whenfunc is a descriptor (such as a normal Python function,classmethod(),staticmethod(),abstractmethod() oranother instance ofpartialmethod), calls to__get__ aredelegated to the underlying descriptor, and an appropriatepartial object returned as the result.

Whenfunc is a non-descriptor callable, an appropriate bound method iscreated dynamically. This behaves like a normal Python function whenused as a method: theself argument will be inserted as the firstpositional argument, even before theargs andkeywords supplied tothepartialmethod constructor.

Example:

>>>classCell:...def__init__(self):...self._alive=False...@property...defalive(self):...returnself._alive...defset_state(self,state):...self._alive=bool(state)...set_alive=partialmethod(set_state,True)...set_dead=partialmethod(set_state,False)...>>>c=Cell()>>>c.aliveFalse>>>c.set_alive()>>>c.aliveTrue

Added in version 3.4.

functools.reduce(function,iterable,[initial,]/)¶

Applyfunction of two arguments cumulatively to the items ofiterable, fromleft to right, so as to reduce the iterable to a single value. For example,reduce(lambdax,y:x+y,[1,2,3,4,5]) calculates((((1+2)+3)+4)+5).The left argument,x, is the accumulated value and the right argument,y, isthe update value from theiterable. If the optionalinitial is present,it is placed before the items of the iterable in the calculation, and serves asa default when the iterable is empty. Ifinitial is not given anditerable contains only one item, the first item is returned.

Roughly equivalent to:

initial_missing=object()defreduce(function,iterable,initial=initial_missing,/):it=iter(iterable)ifinitialisinitial_missing:value=next(it)else:value=initialforelementinit:value=function(value,element)returnvalue

Seeitertools.accumulate() for an iterator that yields all intermediatevalues.

@functools.singledispatch¶

Transform a function into asingle-dispatchgeneric function.

To define a generic function, decorate it with the@singledispatchdecorator. When defining a function using@singledispatch, note that thedispatch happens on the type of the first argument:

>>>fromfunctoolsimportsingledispatch>>>@singledispatch...deffun(arg,verbose=False):...ifverbose:...print("Let me just say,",end=" ")...print(arg)

To add overloaded implementations to the function, use theregister()attribute of the generic function, which can be used as a decorator. Forfunctions annotated with types, the decorator will infer the type of thefirst argument automatically:

>>>@fun.register...def_(arg:int,verbose=False):...ifverbose:...print("Strength in numbers, eh?",end=" ")...print(arg)...>>>@fun.register...def_(arg:list,verbose=False):...ifverbose:...print("Enumerate this:")...fori,eleminenumerate(arg):...print(i,elem)

types.UnionType andtyping.Union can also be used:

>>>@fun.register...def_(arg:int|float,verbose=False):...ifverbose:...print("Strength in numbers, eh?",end=" ")...print(arg)...>>>fromtypingimportUnion>>>@fun.register...def_(arg:Union[list,set],verbose=False):...ifverbose:...print("Enumerate this:")...fori,eleminenumerate(arg):...print(i,elem)...

For code which doesn’t use type annotations, the appropriate typeargument can be passed explicitly to the decorator itself:

>>>@fun.register(complex)...def_(arg,verbose=False):...ifverbose:...print("Better than complicated.",end=" ")...print(arg.real,arg.imag)...

For code that dispatches on a collections type (e.g.,list), but wantsto typehint the items of the collection (e.g.,list[int]), thedispatch type should be passed explicitly to the decorator itself with thetypehint going into the function definition:

>>>@fun.register(list)...def_(arg:list[int],verbose=False):...ifverbose:...print("Enumerate this:")...fori,eleminenumerate(arg):...print(i,elem)

Note

At runtime the function will dispatch on an instance of a list regardlessof the type contained within the list i.e.[1,2,3] will bedispatched the same as["foo","bar","baz"]. The annotationprovided in this example is for static type checkers only and has noruntime impact.

To enable registeringlambdas and pre-existing functions,theregister() attribute can also be used in a functional form:

>>>defnothing(arg,verbose=False):...print("Nothing.")...>>>fun.register(type(None),nothing)

Theregister() attribute returns the undecorated function. Thisenables decorator stacking,pickling, and the creationof unit tests for each variant independently:

>>>@fun.register(float)...@fun.register(Decimal)...deffun_num(arg,verbose=False):...ifverbose:...print("Half of your number:",end=" ")...print(arg/2)...>>>fun_numisfunFalse

When called, the generic function dispatches on the type of the firstargument:

>>>fun("Hello, world.")Hello, world.>>>fun("test.",verbose=True)Let me just say, test.>>>fun(42,verbose=True)Strength in numbers, eh? 42>>>fun(['spam','spam','eggs','spam'],verbose=True)Enumerate this:0 spam1 spam2 eggs3 spam>>>fun(None)Nothing.>>>fun(1.23)0.615

Where there is no registered implementation for a specific type, itsmethod resolution order is used to find a more generic implementation.The original function decorated with@singledispatch is registeredfor the baseobject type, which means it is used if no betterimplementation is found.

If an implementation is registered to anabstract base class,virtual subclasses of the base class will be dispatched to thatimplementation:

>>>fromcollections.abcimportMapping>>>@fun.register...def_(arg:Mapping,verbose=False):...ifverbose:...print("Keys & Values")...forkey,valueinarg.items():...print(key,"=>",value)...>>>fun({"a":"b"})a => b

To check which implementation the generic function will choose fora given type, use thedispatch() attribute:

>>>fun.dispatch(float)<function fun_num at 0x1035a2840>>>>fun.dispatch(dict)# note: default implementation<function fun at 0x103fe0000>

To access all registered implementations, use the read-onlyregistryattribute:

>>>fun.registry.keys()dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,          <class 'decimal.Decimal'>, <class 'list'>,          <class 'float'>])>>>fun.registry[float]<function fun_num at 0x1035a2840>>>>fun.registry[object]<function fun at 0x103fe0000>

Added in version 3.4.

Changed in version 3.7:Theregister() attribute now supports using type annotations.

Changed in version 3.11:Theregister() attribute now supportstypes.UnionTypeandtyping.Union as type annotations.

classfunctools.singledispatchmethod(func)¶

Transform a method into asingle-dispatchgeneric function.

To define a generic method, decorate it with the@singledispatchmethoddecorator. When defining a function using@singledispatchmethod, notethat the dispatch happens on the type of the first non-self or non-clsargument:

classNegator:@singledispatchmethoddefneg(self,arg):raiseNotImplementedError("Cannot negate a")@neg.registerdef_(self,arg:int):return-arg@neg.registerdef_(self,arg:bool):returnnotarg

@singledispatchmethod supports nesting with other decorators such as@classmethod. Note that to allow fordispatcher.register,singledispatchmethod must be theouter mostdecorator. Here is theNegator class with theneg methods bound tothe class, rather than an instance of the class:

classNegator:@singledispatchmethod@classmethoddefneg(cls,arg):raiseNotImplementedError("Cannot negate a")@neg.register@classmethoddef_(cls,arg:int):return-arg@neg.register@classmethoddef_(cls,arg:bool):returnnotarg

The same pattern can be used for other similar decorators:@staticmethod,@abstractmethod, and others.

Added in version 3.8.

functools.update_wrapper(wrapper,wrapped,assigned=WRAPPER_ASSIGNMENTS,updated=WRAPPER_UPDATES)¶

Update awrapper function to look like thewrapped function. The optionalarguments are tuples to specify which attributes of the original function areassigned directly to the matching attributes on the wrapper function and whichattributes of the wrapper function are updated with the corresponding attributesfrom the original function. The default values for these arguments are themodule level constantsWRAPPER_ASSIGNMENTS (which assigns to the wrapperfunction’s__module__,__name__,__qualname__,__annotations__,__type_params__, and__doc__, thedocumentation string) andWRAPPER_UPDATES (which updates the wrapperfunction’s__dict__, i.e. the instance dictionary).

To allow access to the original function for introspection and other purposes(e.g. bypassing a caching decorator such aslru_cache()), this functionautomatically adds a__wrapped__ attribute to the wrapper that refers tothe function being wrapped.

The main intended use for this function is indecorator functions whichwrap the decorated function and return the wrapper. If the wrapper function isnot updated, the metadata of the returned function will reflect the wrapperdefinition rather than the original function definition, which is typically lessthan helpful.

update_wrapper() may be used with callables other than functions. Anyattributes named inassigned orupdated that are missing from the objectbeing wrapped are ignored (i.e. this function will not attempt to set themon the wrapper function).AttributeError is still raised if thewrapper function itself is missing any attributes named inupdated.

Changed in version 3.2:The__wrapped__ attribute is now automatically added.The__annotations__ attribute is now copied by default.Missing attributes no longer trigger anAttributeError.

Changed in version 3.4:The__wrapped__ attribute now always refers to the wrappedfunction, even if that function defined a__wrapped__ attribute.(seebpo-17482)

Changed in version 3.12:The__type_params__ attribute is now copied by default.

@functools.wraps(wrapped,assigned=WRAPPER_ASSIGNMENTS,updated=WRAPPER_UPDATES)¶

This is a convenience function for invokingupdate_wrapper() as afunction decorator when defining a wrapper function. It is equivalent topartial(update_wrapper,wrapped=wrapped,assigned=assigned,updated=updated).For example:

>>>fromfunctoolsimportwraps>>>defmy_decorator(f):...@wraps(f)...defwrapper(*args,**kwds):...print('Calling decorated function')...returnf(*args,**kwds)...returnwrapper...>>>@my_decorator...defexample():..."""Docstring"""...print('Called example function')...>>>example()Calling decorated functionCalled example function>>>example.__name__'example'>>>example.__doc__'Docstring'

Without the use of this decorator factory, the name of the example functionwould have been'wrapper', and the docstring of the originalexample()would have been lost.