Chaining comparison operators:

>>> x = 5>>> 1 < x < 10True>>> 10 < x < 20 False>>> x < 10 < x*10 < 100True>>> 10 > x <= 9True>>> 5 == x > 4True

In case you're thinking it's doing1 < x, which comes out asTrue, and then comparingTrue < 10, which is alsoTrue, then no, that's really not what happens (see the last example.) It's really translating into1 < x and x < 10, andx < 10 and 10 < x * 10 and x*10 < 100, but with less typing and each term is only evaluated once.

Get the python regex parse tree to debug your regex.

Regular expressions are a great feature of python, but debugging them can be a pain, and it's all too easy to get a regex wrong.

Fortunately, python can print the regex parse tree, by passing the undocumented, experimental, hidden flagre.DEBUG (actually, 128) tore.compile.

>>> re.compile("^\[font(?:=(?P<size>[-+][0-9]{1,2}))?\](.*?)[/font]",    re.DEBUG)at at_beginningliteral 91literal 102literal 111literal 110literal 116max_repeat 0 1  subpattern None    literal 61    subpattern 1      in        literal 45        literal 43      max_repeat 1 2        in          range (48, 57)literal 93subpattern 2  min_repeat 0 65535    any Nonein  literal 47  literal 102  literal 111  literal 110  literal 116

Once you understand the syntax, you can spot your errors. There we can see that I forgot to escape the[] in[/font].

Of course you can combine it with whatever flags you want, like commented regexes:

>>> re.compile(""" ^              # start of a line \[font         # the font tag (?:=(?P<size>  # optional [font=+size] [-+][0-9]{1,2} # size specification ))? \]             # end of tag (.*?)          # text between the tags \[/font\]      # end of the tag """, re.DEBUG|re.VERBOSE|re.DOTALL)

enumerate

Wrap an iterable with enumerate and it will yield the item along with its index.

For example:

>>> a = ['a', 'b', 'c', 'd', 'e']>>> for index, item in enumerate(a): print index, item...0 a1 b2 c3 d4 e>>>

References:

• Python tutorial—looping techniques
• Python docs—built-in functions—enumerate
• PEP 279

Creating generators objects

If you write

x=(n for n in foo if bar(n))

you can get out the generator and assign it to x. Now it means you can do

for n in x:

The advantage of this is that you don't need intermediate storage, which you would need if you did

x = [n for n in foo if bar(n)]

In some cases this can lead to significant speed up.

You can append many if statements to the end of the generator, basically replicating nested for loops:

>>> n = ((a,b) for a in range(0,2) for b in range(4,6))>>> for i in n:...   print i (0, 4)(0, 5)(1, 4)(1, 5)

iter() can take a callable argument

For instance:

def seek_next_line(f):    for c in iter(lambda: f.read(1),'\n'):        pass

Theiter(callable, until_value) function repeatedly callscallable and yields its result untiluntil_value is returned.

Be careful with mutable default arguments

>>> def foo(x=[]):...     x.append(1)...     print x... >>> foo()[1]>>> foo()[1, 1]>>> foo()[1, 1, 1]

Instead, you should use a sentinel value denoting "not given" and replace with the mutable you'd like as default:

>>> def foo(x=None):...     if x is None:...         x = []...     x.append(1)...     print x>>> foo()[1]>>> foo()[1]

Sending values into generator functions. For example having this function:

def mygen():    """Yield 5 until something else is passed back via send()"""    a = 5    while True:        f = (yield a) #yield a and possibly get f in return        if f is not None:             a = f  #store the new value

You can:

>>> g = mygen()>>> g.next()5>>> g.next()5>>> g.send(7)  #we send this back to the generator7>>> g.next() #now it will yield 7 until we send something else7

If you don't like using whitespace to denote scopes, you can use the C-style {} by issuing:

from __future__ import braces

The step argument in slice operators. For example:

a = [1,2,3,4,5]>>> a[::2]  # iterate over the whole list in 2-increments[1,3,5]

The special casex[::-1] is a useful idiom for 'x reversed'.

>>> a[::-1][5,4,3,2,1]

Decorators

Decorators allow to wrap a function or method in another function that can add functionality, modify arguments or results, etc. You write decorators one line above the function definition, beginning with an "at" sign (@).

Example shows aprint_args decorator that prints the decorated function's arguments before calling it:

>>> def print_args(function):>>>     def wrapper(*args, **kwargs):>>>         print 'Arguments:', args, kwargs>>>         return function(*args, **kwargs)>>>     return wrapper>>> @print_args>>> def write(text):>>>     print text>>> write('foo')Arguments: ('foo',) {}foo

The for...else syntax (seehttp://docs.python.org/ref/for.html )

for i in foo:    if i == 0:        breakelse:    print("i was never 0")

The "else" block will be normally executed at the end of the for loop, unless the break is called.

The above code could be emulated as follows:

found = Falsefor i in foo:    if i == 0:        found = True        breakif not found:     print("i was never 0")

From 2.5 onwards dicts have a special method__missing__ that is invoked for missing items:

>>> class MyDict(dict):...  def __missing__(self, key):...   self[key] = rv = []...   return rv... >>> m = MyDict()>>> m["foo"].append(1)>>> m["foo"].append(2)>>> dict(m){'foo': [1, 2]}

There is also a dict subclass incollections calleddefaultdict that does pretty much the same but calls a function without arguments for not existing items:

>>> from collections import defaultdict>>> m = defaultdict(list)>>> m["foo"].append(1)>>> m["foo"].append(2)>>> dict(m){'foo': [1, 2]}

I recommend converting such dicts to regular dicts before passing them to functions that don't expect such subclasses. A lot of code usesd[a_key] and catches KeyErrors to check if an item exists which would add a new item to the dict.

In-place value swapping

>>> a = 10>>> b = 5>>> a, b(10, 5)>>> a, b = b, a>>> a, b(5, 10)

The right-hand side of the assignment is an expression that creates a new tuple. The left-hand side of the assignment immediately unpacks that (unreferenced) tuple to the namesa andb.

After the assignment, the new tuple is unreferenced and marked for garbage collection, and the values bound toa andb have been swapped.

As noted in thePython tutorial section on data structures,

Note that multiple assignment is really just a combination of tuple packing and sequence unpacking.

Readable regular expressions

In Python you can split a regular expression over multiple lines, name your matches and insert comments.

Example verbose syntax (fromDive into Python):

>>> pattern = """... ^                   # beginning of string... M{0,4}              # thousands - 0 to 4 M's... (CM|CD|D?C{0,3})    # hundreds - 900 (CM), 400 (CD), 0-300 (0 to 3 C's),...                     #            or 500-800 (D, followed by 0 to 3 C's)... (XC|XL|L?X{0,3})    # tens - 90 (XC), 40 (XL), 0-30 (0 to 3 X's),...                     #        or 50-80 (L, followed by 0 to 3 X's)... (IX|IV|V?I{0,3})    # ones - 9 (IX), 4 (IV), 0-3 (0 to 3 I's),...                     #        or 5-8 (V, followed by 0 to 3 I's)... $                   # end of string... """>>> re.search(pattern, 'M', re.VERBOSE)

Example naming matches (fromRegular Expression HOWTO)

>>> p = re.compile(r'(?P<word>\b\w+\b)')>>> m = p.search( '(((( Lots of punctuation )))' )>>> m.group('word')'Lots'

You can also verbosely write a regex without usingre.VERBOSE thanks to string literal concatenation.

>>> pattern = (...     "^"                 # beginning of string...     "M{0,4}"            # thousands - 0 to 4 M's...     "(CM|CD|D?C{0,3})"  # hundreds - 900 (CM), 400 (CD), 0-300 (0 to 3 C's),...                         #            or 500-800 (D, followed by 0 to 3 C's)...     "(XC|XL|L?X{0,3})"  # tens - 90 (XC), 40 (XL), 0-30 (0 to 3 X's),...                         #        or 50-80 (L, followed by 0 to 3 X's)...     "(IX|IV|V?I{0,3})"  # ones - 9 (IX), 4 (IV), 0-3 (0 to 3 I's),...                         #        or 5-8 (V, followed by 0 to 3 I's)...     "$"                 # end of string... )>>> print pattern"^M{0,4}(CM|CD|D?C{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})$"

Function argument unpacking

You can unpack a list or a dictionary as function arguments using* and**.

For example:

def draw_point(x, y):    # do some magicpoint_foo = (3, 4)point_bar = {'y': 3, 'x': 2}draw_point(*point_foo)draw_point(**point_bar)

Very useful shortcut since lists, tuples and dicts are widely used as containers.

ROT13 is a valid encoding for source code, when you use the right coding declaration at the top of the code file:

#!/usr/bin/env python# -*- coding: rot13 -*-cevag "Uryyb fgnpxbiresybj!".rapbqr("rot13")

Creating new types in a fully dynamic manner

>>> NewType = type("NewType", (object,), {"x": "hello"})>>> n = NewType()>>> n.x"hello"

which is exactly the same as

>>> class NewType(object):>>>     x = "hello">>> n = NewType()>>> n.x"hello"

Probably not the most useful thing, but nice to know.

Edit: Fixed name of new type, should beNewType to be the exact same thing as withclass statement.

Edit: Adjusted the title to more accurately describe the feature.

Context managers and the "with" Statement

Introduced inPEP 343, acontext manager is an object that acts as a run-time context for a suite of statements.

Since the feature makes use of new keywords, it is introduced gradually: it is available in Python 2.5 via the__future__ directive. Python 2.6 and above (including Python 3) has it available by default.

I have used the"with" statement a lot because I think it's a very useful construct, here is a quick demo:

from __future__ import with_statementwith open('foo.txt', 'w') as f:    f.write('hello!')

What's happening here behind the scenes, is that the"with" statement calls the special__enter__ and__exit__ methods on the file object. Exception details are also passed to__exit__ if any exception was raised from the with statement body, allowing for exception handling to happen there.

What this does for you in this particular case is that it guarantees that the file is closed when execution falls out of scope of thewith suite, regardless if that occurs normally or whether an exception was thrown. It is basically a way of abstracting away common exception-handling code.

Other common use cases for this include locking with threads and database transactions.

Dictionaries have a get() method

Dictionaries have a 'get()' method. If you do d['key'] and key isn't there, you get an exception. If you do d.get('key'), you get back None if 'key' isn't there. You can add a second argument to get that item back instead of None, eg: d.get('key', 0).

It's great for things like adding up numbers:

sum[value] = sum.get(value, 0) + 1

Descriptors

They're the magic behind a whole bunch of core Python features.

When you use dotted access to look up a member (eg, x.y), Python first looks for the member in the instance dictionary. If it's not found, it looks for it in the class dictionary. If it finds it in the class dictionary, and the object implements the descriptor protocol, instead of just returning it, Python executes it. A descriptor is any class that implements the__get__,__set__, or__delete__ methods.

Here's how you'd implement your own (read-only) version of property using descriptors:

class Property(object):    def __init__(self, fget):        self.fget = fget    def __get__(self, obj, type):        if obj is None:            return self        return self.fget(obj)

and you'd use it just like the built-in property():

class MyClass(object):    @Property    def foo(self):        return "Foo!"

Descriptors are used in Python to implement properties, bound methods, static methods, class methods and slots, amongst other things. Understanding them makes it easy to see why a lot of things that previously looked like Python 'quirks' are the way they are.

Raymond Hettinger hasan excellent tutorial that does a much better job of describing them than I do.

Conditional Assignment

x = 3 if (y == 1) else 2

It does exactly what it sounds like: "assign 3 to x if y is 1, otherwise assign 2 to x". Note that the parens are not necessary, but I like them for readability. You can also chain it if you have something more complicated:

x = 3 if (y == 1) else 2 if (y == -1) else 1

Though at a certain point, it goes a little too far.

Note that you can use if ... else in any expression. For example:

(func1 if y == 1 else func2)(arg1, arg2)

Here func1 will be called if y is 1 and func2, otherwise. In both cases the corresponding function will be called with arguments arg1 and arg2.

Analogously, the following is also valid:

x = (class1 if y == 1 else class2)(arg1, arg2)

where class1 and class2 are two classes.

Doctest: documentation and unit-testing at the same time.

Example extracted from the Python documentation:

def factorial(n):    """Return the factorial of n, an exact integer >= 0.    If the result is small enough to fit in an int, return an int.    Else return a long.    >>> [factorial(n) for n in range(6)]    [1, 1, 2, 6, 24, 120]    >>> factorial(-1)    Traceback (most recent call last):        ...    ValueError: n must be >= 0    Factorials of floats are OK, but the float must be an exact integer:    """    import math    if not n >= 0:        raise ValueError("n must be >= 0")    if math.floor(n) != n:        raise ValueError("n must be exact integer")    if n+1 == n:  # catch a value like 1e300        raise OverflowError("n too large")    result = 1    factor = 2    while factor <= n:        result *= factor        factor += 1    return resultdef _test():    import doctest    doctest.testmod()    if __name__ == "__main__":    _test()

Named formatting

% -formatting takes a dictionary (also applies %i/%s etc. validation).

>>> print "The %(foo)s is %(bar)i." % {'foo': 'answer', 'bar':42}The answer is 42.>>> foo, bar = 'question', 123>>> print "The %(foo)s is %(bar)i." % locals()The question is 123.

And since locals() is also a dictionary, you can simply pass that as a dict and have % -substitions from your local variables. I think this is frowned upon, but simplifies things..

New Style Formatting

>>> print("The {foo} is {bar}".format(foo='answer', bar=42))

To add more python modules (espcially 3rd party ones), most people seem to use PYTHONPATH environment variables or they add symlinks or directories in their site-packages directories. Another way, is to use *.pth files. Here's the official python doc's explanation:

"The most convenient way [to modify python's search path] is to add a path configuration file to a directory that's already on Python's path, usually to the .../site-packages/ directory. Path configuration files have an extension of .pth, and each line must contain a single path that will be appended to sys.path. (Because the new paths are appended to sys.path, modules in the added directories will not override standard modules. This means you can't use this mechanism for installing fixed versions of standard modules.)"

Exceptionelse clause:

try:  put_4000000000_volts_through_it(parrot)except Voom:  print "'E's pining!"else:  print "This parrot is no more!"finally:  end_sketch()

The use of the else clause is better than adding additional code to the try clause because it avoids accidentally catching an exception that wasn’t raised by the code being protected by the try ... except statement.

Seehttp://docs.python.org/tut/node10.html

Re-raising exceptions:

# Python 2 syntaxtry:    some_operation()except SomeError, e:    if is_fatal(e):        raise    handle_nonfatal(e)# Python 3 syntaxtry:    some_operation()except SomeError as e:    if is_fatal(e):        raise    handle_nonfatal(e)

The 'raise' statement with no arguments inside an error handler tells Python to re-raise the exceptionwith the original traceback intact, allowing you to say "oh, sorry, sorry, I didn't mean to catch that, sorry, sorry."

If you wish to print, store or fiddle with the original traceback, you can get it with sys.exc_info(), and printing it like Python would is done with the 'traceback' module.

Main messages :)

import this# btw look at this module's source :)

De-cyphered:

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better thanright now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!