10.2.functools — Higher-order functions and operations on callable objects¶

Source code:Lib/functools.py

Thefunctools module is for higher-order functions: functions that act onor return other functions. In general, any callable object can be treated as afunction for the purposes of this module.

Thefunctools module defines the following functions:

functools.cmp_to_key(func)¶

Transform an old-style comparison function to akey function. Usedwith tools that accept key functions (such assorted(),min(),max(),heapq.nlargest(),heapq.nsmallest(),itertools.groupby()). This function is primarily used as a transitiontool for programs being converted from Python 2 which supported the use ofcomparison functions.

A comparison function is any callable that accept two arguments, compares them,and returns a negative number for less-than, zero for equality, or a positivenumber for greater-than. A key function is a callable that accepts oneargument and returns another value to be used as the sort key.

Example:

sorted(iterable,key=cmp_to_key(locale.strcoll))# locale-aware sort order

For sorting examples and a brief sorting tutorial, seeSorting HOW TO.

New in version 3.2.

@functools.lru_cache(maxsize=128,typed=False)¶

Decorator to wrap a function with a memoizing callable that saves up to themaxsize most recent calls. It can save time when an expensive or I/O boundfunction is periodically called with the same arguments.

Since a dictionary is used to cache results, the positional and keywordarguments to the function must be hashable.

Ifmaxsize is set toNone, the LRU feature is disabled and the cache cangrow without bound. The LRU feature performs best whenmaxsize is apower-of-two.

Iftyped is set to true, function arguments of different types will becached separately. For example,f(3) andf(3.0) will be treatedas distinct calls with distinct results.

To help measure the effectiveness of the cache and tune themaxsizeparameter, the wrapped function is instrumented with acache_info()function that returns anamed tuple showinghits,misses,maxsize andcurrsize. In a multi-threaded environment, the hitsand misses are approximate.

The decorator also provides acache_clear() function for clearing orinvalidating the cache.

The original underlying function is accessible through the__wrapped__ attribute. This is useful for introspection, forbypassing the cache, or for rewrapping the function with a different cache.

AnLRU (least recently used) cache worksbest when the most recent calls are the best predictors of upcoming calls (forexample, the most popular articles on a news server tend to change each day).The cache’s size limit assures that the cache does not grow without bound onlong-running processes such as web servers.

In general, the LRU cache should only be used when you want to reusepreviously computed values. Accordingly, it doesn’t make sense to cachefunctions with side-effects, functions that need to create distinct mutableobjects on each call, or impure functions such as time() or random().

Example of an LRU cache for static web content:

@lru_cache(maxsize=32)defget_pep(num):'Retrieve text of a Python Enhancement Proposal'resource='http://www.python.org/dev/peps/pep-%04d/'%numtry:withurllib.request.urlopen(resource)ass:returns.read()excepturllib.error.HTTPError:return'Not Found'>>>fornin8,290,308,320,8,218,320,279,289,320,9991:...pep=get_pep(n)...print(n,len(pep))>>>get_pep.cache_info()CacheInfo(hits=3,misses=8,maxsize=32,currsize=8)

Example of efficiently computingFibonacci numbersusing a cache to implement adynamic programmingtechnique:

@lru_cache(maxsize=None)deffib(n):ifn<2:returnnreturnfib(n-1)+fib(n-2)>>>[fib(n)forninrange(16)][0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,610]>>>fib.cache_info()CacheInfo(hits=28,misses=16,maxsize=None,currsize=16)

New in version 3.2.

Changed in version 3.3:Added thetyped option.

@functools.total_ordering¶

Given a class defining one or more rich comparison ordering methods, thisclass decorator supplies the rest. This simplifies the effort involvedin specifying all of the possible rich comparison operations:

The class must define one of__lt__(),__le__(),__gt__(), or__ge__().In addition, the class should supply an__eq__() method.

For example:

@total_orderingclassStudent:def_is_valid_operand(self,other):return(hasattr(other,"lastname")andhasattr(other,"firstname"))def__eq__(self,other):ifnotself._is_valid_operand(other):returnNotImplementedreturn((self.lastname.lower(),self.firstname.lower())==(other.lastname.lower(),other.firstname.lower()))def__lt__(self,other):ifnotself._is_valid_operand(other):returnNotImplementedreturn((self.lastname.lower(),self.firstname.lower())<(other.lastname.lower(),other.firstname.lower()))

Note

While this decorator makes it easy to create well behaved totallyordered types, itdoes come at the cost of slower execution andmore complex stack traces for the derived comparison methods. Ifperformance benchmarking indicates this is a bottleneck for a givenapplication, implementing all six rich comparison methods instead islikely to provide an easy speed boost.

New in version 3.2.

Changed in version 3.4:Returning NotImplemented from the underlying comparison function forunrecognised types is now supported.

functools.partial(func,*args,**keywords)¶

Return a newpartial object which when calledwill behave likefunc called with the positional argumentsargsand keyword argumentskeywords. If more arguments are supplied to thecall, they are appended toargs. If additional keyword arguments aresupplied, they extend and overridekeywords.Roughly equivalent to:

defpartial(func,*args,**keywords):defnewfunc(*fargs,**fkeywords):newkeywords=keywords.copy()newkeywords.update(fkeywords)returnfunc(*args,*fargs,**newkeywords)newfunc.func=funcnewfunc.args=argsnewfunc.keywords=keywordsreturnnewfunc

Thepartial() is used for partial function application which “freezes”some portion of a function’s arguments and/or keywords resulting in a new objectwith a simplified signature. For example,partial() can be used to createa callable that behaves like theint() function where thebase argumentdefaults to two:

>>>fromfunctoolsimportpartial>>>basetwo=partial(int,base=2)>>>basetwo.__doc__='Convert base 2 string to an int.'>>>basetwo('10010')18

classfunctools.partialmethod(func,*args,**keywords)¶

Return a newpartialmethod descriptor which behaveslikepartial except that it is designed to be used as a methoddefinition rather than being directly callable.

func must be adescriptor or a callable (objects which are both,like normal functions, are handled as descriptors).

Whenfunc is a descriptor (such as a normal Python function,classmethod(),staticmethod(),abstractmethod() oranother instance ofpartialmethod), calls to__get__ aredelegated to the underlying descriptor, and an appropriatepartial object returned as the result.

Whenfunc is a non-descriptor callable, an appropriate bound method iscreated dynamically. This behaves like a normal Python function whenused as a method: theself argument will be inserted as the firstpositional argument, even before theargs andkeywords supplied tothepartialmethod constructor.

Example:

>>>classCell(object):...def__init__(self):...self._alive=False...@property...defalive(self):...returnself._alive...defset_state(self,state):...self._alive=bool(state)...set_alive=partialmethod(set_state,True)...set_dead=partialmethod(set_state,False)...>>>c=Cell()>>>c.aliveFalse>>>c.set_alive()>>>c.aliveTrue

New in version 3.4.

functools.reduce(function,iterable[,initializer])¶

Applyfunction of two arguments cumulatively to the items ofsequence, fromleft to right, so as to reduce the sequence to a single value. For example,reduce(lambdax,y:x+y,[1,2,3,4,5]) calculates((((1+2)+3)+4)+5).The left argument,x, is the accumulated value and the right argument,y, isthe update value from thesequence. If the optionalinitializer is present,it is placed before the items of the sequence in the calculation, and serves asa default when the sequence is empty. Ifinitializer is not given andsequence contains only one item, the first item is returned.

Roughly equivalent to:

defreduce(function,iterable,initializer=None):it=iter(iterable)ifinitializerisNone:value=next(it)else:value=initializerforelementinit:value=function(value,element)returnvalue

@functools.singledispatch¶

Transform a function into asingle-dispatchgeneric function.

To define a generic function, decorate it with the@singledispatchdecorator. Note that the dispatch happens on the type of the first argument,create your function accordingly:

>>>fromfunctoolsimportsingledispatch>>>@singledispatch...deffun(arg,verbose=False):...ifverbose:...print("Let me just say,",end=" ")...print(arg)

To add overloaded implementations to the function, use theregister()attribute of the generic function. It is a decorator, taking a typeparameter and decorating a function implementing the operation for thattype:

>>>@fun.register(int)...def_(arg,verbose=False):...ifverbose:...print("Strength in numbers, eh?",end=" ")...print(arg)...>>>@fun.register(list)...def_(arg,verbose=False):...ifverbose:...print("Enumerate this:")...fori,eleminenumerate(arg):...print(i,elem)

To enable registering lambdas and pre-existing functions, theregister() attribute can be used in a functional form:

>>>defnothing(arg,verbose=False):...print("Nothing.")...>>>fun.register(type(None),nothing)

Theregister() attribute returns the undecorated function whichenables decorator stacking, pickling, as well as creating unit tests foreach variant independently:

>>>@fun.register(float)...@fun.register(Decimal)...deffun_num(arg,verbose=False):...ifverbose:...print("Half of your number:",end=" ")...print(arg/2)...>>>fun_numisfunFalse

When called, the generic function dispatches on the type of the firstargument:

>>>fun("Hello, world.")Hello, world.>>>fun("test.",verbose=True)Let me just say, test.>>>fun(42,verbose=True)Strength in numbers, eh? 42>>>fun(['spam','spam','eggs','spam'],verbose=True)Enumerate this:0 spam1 spam2 eggs3 spam>>>fun(None)Nothing.>>>fun(1.23)0.615

Where there is no registered implementation for a specific type, itsmethod resolution order is used to find a more generic implementation.The original function decorated with@singledispatch is registeredfor the baseobject type, which means it is used if no betterimplementation is found.

To check which implementation will the generic function choose fora given type, use thedispatch() attribute:

>>>fun.dispatch(float)<function fun_num at 0x1035a2840>>>>fun.dispatch(dict)# note: default implementation<function fun at 0x103fe0000>

To access all registered implementations, use the read-onlyregistryattribute:

>>>fun.registry.keys()dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,          <class 'decimal.Decimal'>, <class 'list'>,          <class 'float'>])>>>fun.registry[float]<function fun_num at 0x1035a2840>>>>fun.registry[object]<function fun at 0x103fe0000>

New in version 3.4.

functools.update_wrapper(wrapper,wrapped,assigned=WRAPPER_ASSIGNMENTS,updated=WRAPPER_UPDATES)¶

Update awrapper function to look like thewrapped function. The optionalarguments are tuples to specify which attributes of the original function areassigned directly to the matching attributes on the wrapper function and whichattributes of the wrapper function are updated with the corresponding attributesfrom the original function. The default values for these arguments are themodule level constantsWRAPPER_ASSIGNMENTS (which assigns to the wrapperfunction’s__module__,__name__,__qualname__,__annotations__and__doc__, the documentation string) andWRAPPER_UPDATES (whichupdates the wrapper function’s__dict__, i.e. the instance dictionary).

To allow access to the original function for introspection and other purposes(e.g. bypassing a caching decorator such aslru_cache()), this functionautomatically adds a__wrapped__ attribute to the wrapper that refers tothe function being wrapped.

The main intended use for this function is indecorator functions whichwrap the decorated function and return the wrapper. If the wrapper function isnot updated, the metadata of the returned function will reflect the wrapperdefinition rather than the original function definition, which is typically lessthan helpful.

update_wrapper() may be used with callables other than functions. Anyattributes named inassigned orupdated that are missing from the objectbeing wrapped are ignored (i.e. this function will not attempt to set themon the wrapper function).AttributeError is still raised if thewrapper function itself is missing any attributes named inupdated.

New in version 3.2:Automatic addition of the__wrapped__ attribute.

New in version 3.2:Copying of the__annotations__ attribute by default.

Changed in version 3.2:Missing attributes no longer trigger anAttributeError.

Changed in version 3.4:The__wrapped__ attribute now always refers to the wrappedfunction, even if that function defined a__wrapped__ attribute.(seebpo-17482)

@functools.wraps(wrapped,assigned=WRAPPER_ASSIGNMENTS,updated=WRAPPER_UPDATES)¶

This is a convenience function for invokingupdate_wrapper() as afunction decorator when defining a wrapper function. It is equivalent topartial(update_wrapper,wrapped=wrapped,assigned=assigned,updated=updated).For example:

>>>fromfunctoolsimportwraps>>>defmy_decorator(f):...@wraps(f)...defwrapper(*args,**kwds):...print('Calling decorated function')...returnf(*args,**kwds)...returnwrapper...>>>@my_decorator...defexample():..."""Docstring"""...print('Called example function')...>>>example()Calling decorated functionCalled example function>>>example.__name__'example'>>>example.__doc__'Docstring'

Without the use of this decorator factory, the name of the example functionwould have been'wrapper', and the docstring of the originalexample()would have been lost.