Idiomatic Python: boolean expressions
You might think that boolean expressions — most frequently used as conditional guards which are the the bit of code that tests whether anif
orwhile
statement should execute — are a fairly straight-forward concept and that there isn’t really anything subtle to them at all. And while the general concept is simple, there are some idiomatic practices to follow when writing them.
To start off, we should make sure everyone understands what makes something considered true or false (sometimes referred to as being “truthy” or not). The officialdefinition of what is true or false in Python 3 is:
False
,None
, numeric zero of all types, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as being true. User-defined objects can customize their truth value by providing a__bool__()
method.
A bit of Python history: During the discussion ofadding the boolean type in Python 2.3, some people didn’t like that the definition of what was considered false went from “anything that represents emptiness”, to “anything that represents emptinessandFalse
” which some viewed as a loss of simplicity. On the other side people argued thatFalse
helped make code clearer. In the end the side arguing that the concept ofFalse
was larger and thus the side for clearer code won. You might also have noticed that booleans are notthat old in Python, which is why booleans can (mostly) be treated as integers due to backwards-compatibility with code that simply used1
and0
to representTrue
andFalse
, respectively.
The first piece of advice is to not overdo the use ofis
comparisons. Theis
is foridentity comparisons which means it evaluates toTrue
only if both objects involved in the expression are literally the same object (this has nothing to do with value). Unfortunately people can easily end up conflating an identity comparison with a value comparison. For instance, some people accidentally discover that some implementations of Python cache certain values for performance, leading to expressions like:
40 + 2 is 42 # True in CPython, not necessarily in other VMs.
being true. But this caching of numbers isn’t part of the language definition of Python, making it just a quirky side-effect of an implementation detail. This is a problem then if you either change Python implementations or happen to think that usingis
with numbers works with any number, which isn’t true if you try something like:
2**32 is 2**32 # False.
which evaluates toFalse
. In other words, only useis
if you really,really want to test for identity and not value.
Another place where we have seenis
used in a non-idiomatic fashion is directly testing forTrue
orFalse
, e.g.:
something() is False # Too restrictive.
This is technically not wrong like with the previous example becauseFalse
is a singleton — just likeNone
andTrue
— which means there is only one instance ofFalse
to actually compare against. Where this goes astray is it is unnecessarily restrictive. Thanks to Python being a huge proponent ofduck typing, tying down any API specifically to onlyTrue
orFalse
is frowned upon as it locks an API to a specific type. If for some reason the API changed to return values of a different type but has the same boolean interpretation then this code would suddenly break. Instead of directly checking forFalse
, the code should have simply checked for false value:
not something() # Just right.
And this extends to other types as well, so don’t dospam == []
if you care if something is empty, simply donot spam
in case the API that gave you the value forspam
suddenly starts returning tuples instead of lists.
About the only time you might legitimately find the need to useis
in day-to-day code is withNone
. Sometimes you might come across an API whereNone
has special meaning, in which case you should useis None
to check for that specific value. For example, modules in Python have a__package__
attribute which stores a string representing what package the module belongs to. The trick is that top-level modules — i.e., modules that are not contained in a package — have__package__
set to the empty string which is false but is a valid value, but there is a need to have a value represent not knowing what__package__
should be set to. In that instance,None
is used to represent “I don’t know”. This allows thecode that calculates what package a module belongs to to use:
package is None # OK when you need an "I don't know" value.
to detect if the package name isn’t known (not package
would incorrectly think that''
represented that as well). Do make sure to not overuse this kind of use ofNone
, though, as a false value tends to meet the need of representing “I don’t know”.
Another bit of advice is to think twice before defining__bool__()
on our own classes. While you should definitely define the method on classes representing containers (to help with that “empty if false” concept), in all cases you should stop and think about whether it truly makes sense to define the method. While it may be tempting to use__bool__()
to represent some sort of state of an object, the ramifications can be surprisingly far-reaching as it means suddenly people have to start explicitly checking for some special value likeNone
which represents whether an API returned an actual value or not instead of simply relying on all object defaulting to being true. As an example of how defining__bool__()
can be surprising, see thePython issue where there was a multi-year discussion over how definingdatetime.time()
to be false at midnight but true for all other values was a mistake and how best to fix it (in the end the implementation of__bool__()
wasremoved in Python 3.5).
If you find yourself needing to provide a specific default value when faced with a possible false value, usingor
can be helpful. Bothand
andor
don’t return a specific boolean value but the first value that forces a known true value. In the case ofor
this means either the first value if it is true or else the last value no matter what. This means that if you had something like:
# Use the value from something() if it is true, else default to None.spam = something() or None
thenspam
would gain the value fromsomething()
if it was true, else it would be set toNone
. And because bothand
andor
short-circuit, you can combine this with some object instantiation and know that it won’t occur unless necessary;
# Only execute AnotherThing() if something() returns a false value.spam = something() or AnotherThing()
won’t actually executeAnotherThing()
unlesssomething()
returns a false value.
And finally, make sure to useany()
andall()
when possible. These built-in functions are very convenient when they are needed and when combined with generator expressions they are rather powerful.
Author
Dev lead on the Python extension for Visual Studio Code. Python core developer since April 2003.