April 18th, 2016
0 reactions

Idiomatic Python: boolean expressions

Brett Cannon
Principal Software Engineering Manager

You might think that boolean expressions — most frequently used as conditional guards which are the the bit of code that tests whether anif orwhile statement should execute — are a fairly straight-forward concept and that there isn’t really anything subtle to them at all. And while the general concept is simple, there are some idiomatic practices to follow when writing them.

To start off, we should make sure everyone understands what makes something considered true or false (sometimes referred to as being “truthy” or not). The officialdefinition of what is true or false in Python 3 is:

False,None, numeric zero of all types, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as being true. User-defined objects can customize their truth value by providing a__bool__() method.

A bit of Python history: During the discussion ofadding the boolean type in Python 2.3, some people didn’t like that the definition of what was considered false went from “anything that represents emptiness”, to “anything that represents emptinessandFalse” which some viewed as a loss of simplicity. On the other side people argued thatFalse helped make code clearer. In the end the side arguing that the concept ofFalse was larger and thus the side for clearer code won. You might also have noticed that booleans are notthat old in Python, which is why booleans can (mostly) be treated as integers due to backwards-compatibility with code that simply used1 and0 to representTrue andFalse, respectively.

The first piece of advice is to not overdo the use ofis comparisons. Theis is foridentity comparisons which means it evaluates toTrue only if both objects involved in the expression are literally the same object (this has nothing to do with value). Unfortunately people can easily end up conflating an identity comparison with a value comparison. For instance, some people accidentally discover that some implementations of Python cache certain values for performance, leading to expressions like:

40 + 2 is 42  # True in CPython, not necessarily in other VMs.

being true. But this caching of numbers isn’t part of the language definition of Python, making it just a quirky side-effect of an implementation detail. This is a problem then if you either change Python implementations or happen to think that usingis with numbers works with any number, which isn’t true if you try something like:

2**32 is 2**32 # False.

which evaluates toFalse. In other words, only useis if you really,really want to test for identity and not value.

Another place where we have seenis used in a non-idiomatic fashion is directly testing forTrue orFalse, e.g.:

something() is False  # Too restrictive.

This is technically not wrong like with the previous example becauseFalse is a singleton — just likeNone andTrue — which means there is only one instance ofFalse to actually compare against. Where this goes astray is it is unnecessarily restrictive. Thanks to Python being a huge proponent ofduck typing, tying down any API specifically to onlyTrue orFalse is frowned upon as it locks an API to a specific type. If for some reason the API changed to return values of a different type but has the same boolean interpretation then this code would suddenly break. Instead of directly checking forFalse, the code should have simply checked for false value:

not something()  # Just right.

And this extends to other types as well, so don’t dospam == [] if you care if something is empty, simply donot spam in case the API that gave you the value forspam suddenly starts returning tuples instead of lists.

About the only time you might legitimately find the need to useis in day-to-day code is withNone. Sometimes you might come across an API whereNone has special meaning, in which case you should useis None to check for that specific value. For example, modules in Python have a__package__ attribute which stores a string representing what package the module belongs to. The trick is that top-level modules — i.e., modules that are not contained in a package — have__package__ set to the empty string which is false but is a valid value, but there is a need to have a value represent not knowing what__package__ should be set to. In that instance,None is used to represent “I don’t know”. This allows thecode that calculates what package a module belongs to to use:

package is None  # OK when you need an "I don't know" value.

to detect if the package name isn’t known (not package would incorrectly think that'' represented that as well). Do make sure to not overuse this kind of use ofNone, though, as a false value tends to meet the need of representing “I don’t know”.

Another bit of advice is to think twice before defining__bool__() on our own classes. While you should definitely define the method on classes representing containers (to help with that “empty if false” concept), in all cases you should stop and think about whether it truly makes sense to define the method. While it may be tempting to use__bool__() to represent some sort of state of an object, the ramifications can be surprisingly far-reaching as it means suddenly people have to start explicitly checking for some special value likeNone which represents whether an API returned an actual value or not instead of simply relying on all object defaulting to being true. As an example of how defining__bool__() can be surprising, see thePython issue where there was a multi-year discussion over how definingdatetime.time() to be false at midnight but true for all other values was a mistake and how best to fix it (in the end the implementation of__bool__() wasremoved in Python 3.5).

If you find yourself needing to provide a specific default value when faced with a possible false value, usingor can be helpful. Bothand andor don’t return a specific boolean value but the first value that forces a known true value. In the case ofor this means either the first value if it is true or else the last value no matter what. This means that if you had something like:

# Use the value from something() if it is true, else default to None.spam = something() or None

thenspam would gain the value fromsomething() if it was true, else it would be set toNone. And because bothand andor short-circuit, you can combine this with some object instantiation and know that it won’t occur unless necessary;

# Only execute AnotherThing() if something() returns a false value.spam = something() or AnotherThing()

won’t actually executeAnotherThing() unlesssomething() returns a false value.

And finally, make sure to useany() andall() when possible. These built-in functions are very convenient when they are needed and when combined with generator expressions they are rather powerful.

Category
Share

Author

Brett Cannon
Principal Software Engineering Manager

Dev lead on the Python extension for Visual Studio Code. Python core developer since April 2003.

0 comments

Discussion is closed.

    Stay informed

    Get notified when new posts are published.
    Follow this blog
    youtube