Sorting Basics¶

A simple ascending sort is very easy: just call thesorted() function. Itreturns a new sorted list:

>>>sorted([5,2,3,1,4])[1, 2, 3, 4, 5]

You can also use thelist.sort() method of a list. It modifies the listin-place (and returnsNone to avoid confusion). Usually it’s less convenientthansorted() - but if you don’t need the original list, it’s slightlymore efficient.

>>>a=[5,2,3,1,4]>>>a.sort()>>>a[1, 2, 3, 4, 5]

Another difference is that thelist.sort() method is only defined forlists. In contrast, thesorted() function accepts any iterable.

>>>sorted({1:'D',2:'B',3:'B',4:'E',5:'A'})[1, 2, 3, 4, 5]

Key Functions¶

Starting with Python 2.4, bothlist.sort() andsorted() added akey parameter to specify a function to be called on each list element prior tomaking comparisons.

For example, here’s a case-insensitive string comparison:

>>>sorted("This is a test string from Andrew".split(),key=str.lower)['a', 'Andrew', 'from', 'is', 'string', 'test', 'This']

The value of thekey parameter should be a function that takes a single argumentand returns a key to use for sorting purposes. This technique is fast becausethe key function is called exactly once for each input record.

A common pattern is to sort complex objects using some of the object’s indicesas keys. For example:

>>>student_tuples=[...('john','A',15),...('jane','B',12),...('dave','B',10),...]>>>sorted(student_tuples,key=lambdastudent:student[2])# sort by age[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

The same technique works for objects with named attributes. For example:

>>>classStudent:...def__init__(self,name,grade,age):...self.name=name...self.grade=grade...self.age=age...def__repr__(self):...returnrepr((self.name,self.grade,self.age))

>>>student_objects=[...Student('john','A',15),...Student('jane','B',12),...Student('dave','B',10),...]>>>sorted(student_objects,key=lambdastudent:student.age)# sort by age[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

Operator Module Functions¶

The key-function patterns shown above are very common, so Python providesconvenience functions to make accessor functions easier and faster. The operatormodule hasoperator.itemgetter(),operator.attrgetter(), andstarting in Python 2.5 anoperator.methodcaller() function.

Using those functions, the above examples become simpler and faster:

>>>fromoperatorimportitemgetter,attrgetter

>>>sorted(student_tuples,key=itemgetter(2))[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

>>>sorted(student_objects,key=attrgetter('age'))[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

The operator module functions allow multiple levels of sorting. For example, tosort bygrade then byage:

>>>sorted(student_tuples,key=itemgetter(1,2))[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]

>>>sorted(student_objects,key=attrgetter('grade','age'))[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]

Theoperator.methodcaller() function makes method calls with fixedparameters for each object being sorted. For example, thestr.count()method could be used to compute message priority by counting thenumber of exclamation marks in a message:

>>>fromoperatorimportmethodcaller>>>messages=['critical!!!','hurry!','standby','immediate!!']>>>sorted(messages,key=methodcaller('count','!'))['standby', 'hurry!', 'immediate!!', 'critical!!!']

Ascending and Descending¶

Bothlist.sort() andsorted() accept areverse parameter with aboolean value. This is used to flag descending sorts. For example, to get thestudent data in reverseage order:

>>>sorted(student_tuples,key=itemgetter(2),reverse=True)[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]

>>>sorted(student_objects,key=attrgetter('age'),reverse=True)[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]

Sort Stability and Complex Sorts¶

Starting with Python 2.2, sorts are guaranteed to bestable. That means thatwhen multiple records have the same key, their original order is preserved.

>>>data=[('red',1),('blue',1),('red',2),('blue',2)]>>>sorted(data,key=itemgetter(0))[('blue', 1), ('blue', 2), ('red', 1), ('red', 2)]

Notice how the two records forblue retain their original order so that('blue',1) is guaranteed to precede('blue',2).

This wonderful property lets you build complex sorts in a series of sortingsteps. For example, to sort the student data by descendinggrade and thenascendingage, do theage sort first and then sort again usinggrade:

>>>s=sorted(student_objects,key=attrgetter('age'))# sort on secondary key>>>sorted(s,key=attrgetter('grade'),reverse=True)# now sort on primary key, descending[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

TheTimsort algorithm used in Pythondoes multiple sorts efficiently because it can take advantage of any orderingalready present in a dataset.

The Old Way Using Decorate-Sort-Undecorate¶

This idiom is called Decorate-Sort-Undecorate after its three steps:

• First, the initial list is decorated with new values that control the sort order.

• Second, the decorated list is sorted.

• Finally, the decorations are removed, creating a list that contains only theinitial values in the new order.

For example, to sort the student data bygrade using the DSU approach:

>>>decorated=[(student.grade,i,student)fori,studentinenumerate(student_objects)]>>>decorated.sort()>>>[studentforgrade,i,studentindecorated]# undecorate[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]

This idiom works because tuples are compared lexicographically; the first itemsare compared; if they are the same then the second items are compared, and soon.

It is not strictly necessary in all cases to include the indexi in thedecorated list, but including it gives two benefits:

• The sort is stable – if two items have the same key, their order will bepreserved in the sorted list.

• The original items do not have to be comparable because the ordering of thedecorated tuples will be determined by at most the first two items. So forexample the original list could contain complex numbers which cannot be sorteddirectly.

Another name for this idiom isSchwartzian transform,after Randal L. Schwartz, who popularized it among Perl programmers.

For large lists and lists where the comparison information is expensive tocalculate, and Python versions before 2.4, DSU is likely to be the fastest wayto sort the list. For 2.4 and later, key functions provide the samefunctionality.

The Old Way Using thecmp Parameter¶

Many constructs given in this HOWTO assume Python 2.4 or later. Before that,there was nosorted() builtin andlist.sort() took no keywordarguments. Instead, all of the Py2.x versions supported acmp parameter tohandle user specified comparison functions.

In Python 3, thecmp parameter was removed entirely (as part of a larger effort tosimplify and unify the language, eliminating the conflict between richcomparisons and the__cmp__() magic method).

In Python 2,sort() allowed an optional function which can be called for doing thecomparisons. That function should take two arguments to be compared and thenreturn a negative value for less-than, return zero if they are equal, or returna positive value for greater-than. For example, we can do:

>>>defnumeric_compare(x,y):...returnx-y>>>sorted([5,2,4,1,3],cmp=numeric_compare)[1, 2, 3, 4, 5]

Or you can reverse the order of comparison with:

>>>defreverse_numeric(x,y):...returny-x>>>sorted([5,2,4,1,3],cmp=reverse_numeric)[5, 4, 3, 2, 1]

When porting code from Python 2.x to 3.x, the situation can arise when you havethe user supplying a comparison function and you need to convert that to a keyfunction. The following wrapper makes that easy to do:

defcmp_to_key(mycmp):'Convert a cmp= function into a key= function'classK(object):def__init__(self,obj,*args):self.obj=objdef__lt__(self,other):returnmycmp(self.obj,other.obj)<0def__gt__(self,other):returnmycmp(self.obj,other.obj)>0def__eq__(self,other):returnmycmp(self.obj,other.obj)==0def__le__(self,other):returnmycmp(self.obj,other.obj)<=0def__ge__(self,other):returnmycmp(self.obj,other.obj)>=0def__ne__(self,other):returnmycmp(self.obj,other.obj)!=0returnK

To convert to a key function, just wrap the old comparison function:

>>>sorted([5,2,4,1,3],key=cmp_to_key(reverse_numeric))[5, 4, 3, 2, 1]

In Python 2.7, thefunctools.cmp_to_key() function was added to thefunctools module.

Odd and Ends¶

• For locale aware sorting, uselocale.strxfrm() for a key function orlocale.strcoll() for a comparison function.

• Thereverse parameter still maintains sort stability (so that records withequal keys retain their original order). Interestingly, that effect can besimulated without the parameter by using the builtinreversed() functiontwice:

  >>>data=[('red',1),('blue',1),('red',2),('blue',2)]>>>standard_way=sorted(data,key=itemgetter(0),reverse=True)>>>double_reversed=list(reversed(sorted(reversed(data),key=itemgetter(0))))>>>assertstandard_way==double_reversed>>>standard_way[('red', 1), ('red', 2), ('blue', 1), ('blue', 2)]

• To create a standard sort order for a class, just add the appropriate richcomparison methods:

  >>>Student.__eq__=lambdaself,other:self.age==other.age>>>Student.__ne__=lambdaself,other:self.age!=other.age>>>Student.__lt__=lambdaself,other:self.age<other.age>>>Student.__le__=lambdaself,other:self.age<=other.age>>>Student.__gt__=lambdaself,other:self.age>other.age>>>Student.__ge__=lambdaself,other:self.age>=other.age>>>sorted(student_objects)[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]

  For general purpose comparisons, the recommended approach is to define all sixrich comparison operators. Thefunctools.total_ordering() classdecorator makes this easy to implement.

• Key functions need not depend directly on the objects being sorted. A keyfunction can also access external resources. For instance, if the student gradesare stored in a dictionary, they can be used to sort a separate list of studentnames:

  >>>students=['dave','john','jane']>>>grades={'john':'F','jane':'A','dave':'C'}>>>sorted(students,key=grades.__getitem__)['jane', 'dave', 'john']