"Data! Data! Data!" he cried impatiently. "I can't make bricks without clay."

– Sherlock Holmes, in The Adventure of the Copper Beeches

Everything you do with a computer is managing data. Data comes in many different shapes and flavors. It's the music you listen to, the movies you stream, the PDFs you open. Even the source of the chapter you're reading at this very moment is just a file, which is data.

Data can be simple, whether it is an integer number to represent an age, or complex, like an order placed on a website. It can be about a single object or about a collection of them. Data can even be about data—that is, metadata. This is data that describes the design of other data structures, or data that describes application data or its context. In Python,objects are our abstraction for data, and Python has an amazing variety of data structures that you can use to represent data or combine them to create your own custom data.

In this chapter, we are going to cover the following:

• Python objects' structures
• Mutability and immutability
• Built-in data types: numbers, strings, dates and times, sequences, collections, and mapping types
• Thecollections module
• Enumerations

Before we delve into the specifics, we want you to be very clear about objects in Python, so let's talk a little bit more about them. As we already said,everything in Python is an object. But what really happens when you type an instruction likeage = 42 in a Python module?

So, whathappens is that anobject is created. It gets anid, thetype is set toint (integer number), and thevalue to 42. A name,age, is placed in the global namespace, pointing to that object. Therefore, whenever we are in the global namespace, after the execution of that line, we can retrieve that object by simply accessing it through its name:age.

If you were to move house, you would put all the knives, forks, and spoons in a box and label itcutlery. This is exactly the same concept. Here is a screenshot of what it may look like (you may have to tweak the settings to get to the same view):

Figure 2.1: A name pointing to an object

So, for the rest of this chapter, whenever you read something such asname = some_value, think of a name placed in the namespace that is tied to the scope in which the instruction was written, with a nice arrow pointing to an object that has anid, atype, and avalue. There is a little bit more to say about this mechanism, but it's much easier to talk about it using an example, so we'll come back to this later.

The first fundamental distinction that Python makes on data is about whether or not the value of an object can change. If the value can change, the object is calledmutable, whereas if the value cannot change, the objectis calledimmutable.

It is very important that you understand the distinction between mutable and immutable because it affects the code you write; take this example:

>>> age =42>>> age42>>> age =43#A>>> age43

In the preceding code, on line#A, have we changed the value ofage? Well, no. But now it's 43 (we hear what you are saying...). Yes, it's 43, but 42 was an integer number, of the typeint, which is immutable. So, what happened is really that on the first line,age is a name that is set to point to anint object, whose value is 42. When we typeage = 43, what happens is that another object is created, of the typeint and value 43 (also, theid will be different), and the nameage is set to point to it. So, in fact, we did not change that 42 to 43—we actually just pointedage to a different location, which is the newint object whose value is 43. Let's see the same code also printing the IDs:

>>> age =42>>>id(age)4377553168>>> age =43>>>id(age)4377553200

Notice that we print the IDs by calling the built-inid() function. As you can see, they are different, as expected. Bear in mind thatage points to one object at a time:42 first, then43—never together.

Now, let's see the same example using amutable object. For this example, let's just use aPerson object, that has a propertyage (don't worry about the class declaration for now—it is there only for completeness):

>>>classPerson:...def__init__(self, age):...         self.age = age...>>> fab = Person(age=42)>>> fab.age42>>>id(fab)4380878496>>>id(fab.age)4377553168>>> fab.age =25# I wish!>>>id(fab)# will be the same4380878496>>>id(fab.age)# will be different4377552624

In this case, we set up an objectfab whosetype isPerson (a custom class). On creation, the object is given the age of 42. We then print it, along with the object ID, and the ID ofage as well. Notice that, even after we changeage to be 25, the ID offab stays the same (while the ID ofage has changed, of course). Custom objects in Python are mutable (unless you code them not to be). Keep this concept in mind, as it's very important. We'll remind you about it throughout the rest of the chapter.

Let's start by exploring Python's built-in data types for numbers. Python was designed by a man with a master's degree in mathematics and computer science, so it's only logical that it has amazingsupport for numbers.

Numbers are immutable objects.

Integers

Python integers have an unlimited range, subject only to the available virtual memory. This means that it doesn't really matter how big a number you want to store is—as long as it can fit in your computer's memory, Python will take care of it.

Integer numbers can be positive, negative, or 0 (zero). They support all the basic mathematical operations, as shown in the following example:

>>> a =14>>> b =3>>> a + b# addition17>>> a - b# subtraction11>>> a * b# multiplication42>>> a / b# true division4.666666666666667>>> a // b# integer division4>>> a % b# modulo operation (reminder of division)2>>> a ** b# power operation2744

The preceding code should be easy to understand. Just notice one important thing: Python has two division operators, one performs theso-calledtrue division (/), which returns the quotient of the operands, and another one, the so-calledinteger division (//), which returns thefloored quotient of the operands.

Let's see how division behaves differently when we introduce negative numbers:

>>>7 /4# true division1.75>>>7 //4# integer division, truncation returns 11>>> -7 /4# true division again, result is opposite of previous-1.75>>> -7 //4# integer div., result not the opposite of previous-2

This is an interesting example. If you were expecting a -1 on the last line, don't feel bad, it's just the way Pythonworks. Integer division in Python isalways rounded toward minus infinity. If, instead of flooring, you want to truncate a number to an integer, you can use the built-inint() function, as shown in the following example:

>>>int(1.75)1>>>int(-1.75)-1

Notice that the truncation is done toward0.

>>> int('10110', base=2)

It's worth noting that the power operator,**, also has a built-in function counterpart,pow(), shown in the example below:

>>>pow(10,3)1000.0  # result is float>>>10 **31000  # result is int>>>pow(10, -3)0.001>>>10 ** -30.001

There is also an operator tocalculate the remainder of a division. It's called themodulo operator, and it's represented by a percentage symbol (%):

>>>10 %3# remainder of the division 10 // 31>>>10 %4# remainder of the division 10 // 42

Thepow() function allows a third argument to performmodular exponentiation. The form with three arguments now accepts a negative exponent in the case where the base is relatively prime to the modulus.

The result is themodular multiplicative inverse of thebase (or a suitable power of that, when the exponent is negative, but not -1), modulo the third argument. Here's an example:

>>>pow(123,4)228886641>>>pow(123,4,100)41  # notice: 228886641 % 100 == 41>>>pow(37, -1,43)# modular inverse of 37 mod 437>>>7 *37 %43# proof the above is correct1

One nice feature introduced in Python 3.6 is the ability to add underscores within number literals (between digits or base specifiers, but not leading or trailing). The purpose is to help make some numbers more readable, such as1_000_000_000:

>>> n =1_024>>> n1024>>> hex_n =0x_4_0_0# 0x400 == 1024>>> hex_n1024

Booleans

Boolean algebra is thatsubset of algebra in which the values of the variables are the truth values, true and false. In Python,True andFalseare two keywords that are used to represent truth values. Booleans are a subclass of integers, soTrue andFalsebehave respectively like 1 and 0. The equivalent of the int class for Booleans is thebool class, which returns eitherTrue orFalse. Every built-in Python object has a value in the Boolean context, which means they basically evaluate to eitherTrue orFalse when fed to thebool function. We'll see all about this inChapter 3,Conditionals and Iteration.

Boolean values can be combined in Boolean expressions using the logical operatorsand,or, andnot. Again, we'll see them in full in the next chapter, so for now let's just see a simple example:

>>>int(True)# True behaves like 11>>>int(False)# False behaves like 00>>>bool(1)# 1 evaluates to True in a Boolean contextTrue>>>bool(-42)# and so does every non-zero numberTrue>>>bool(0)# 0 evaluates to FalseFalse>>># quick peek at the operators (and, or, not)>>>notTrueFalse>>>notFalseTrue>>>TrueandTrueTrue>>>FalseorTrueTrue

You cansee thatTrue andFalse are subclasses of integers when you try to add them. Python upcasts them to integers and performs the addition:

>>>1 +True2>>>False +4242>>>7 -True6

Real numbers

Realnumbers, orfloating point numbers, are represented in Pythonaccording to theIEEE 754 double-precision binary floating point format, which is stored in 64 bits of information divided into three sections: sign, exponent, and mantissa.

Several programming languages give coders two different formats: single and double precision. The former takes up 32 bits of memory, the latter 64. Python supports only the double format. Let's see a simple example:

>>> pi =3.1415926536# how many digits of PI can you remember?>>> radius =4.5>>> area = pi * (radius **2)>>> area63.617251235400005

Thesys.float_info sequence holds information about how floating point numbers will behave on your system. This is an example of what you might see:

>>>import sys>>> sys.float_infosys.float_info(    max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308,    min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307,    dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2,    rounds=1)

Let's make a few considerations here: we have 64 bits to represent floating point numbers. This means we can represent at most2⁶⁴ (that is 18,446,744,073,709,551,616) distinct numbers. Take a look at themax andepsilon values for the float numbers, and you will realize that it's impossible to represent them all. There is just not enough space, so they are approximated to the closest representable number. You probably think that only extremely big or extremely small numbers suffer from this issue. Well, think again and try the following in your console:

>>>0.3 -0.1 *3# this should be 0!!!-5.551115123125783e-17

What does this tell you? It tells you that double precision numbers suffer from approximation issues even when it comes to simple numbers like 0.1 or 0.3. Why is this important? It can be a big problem if you are handling prices, or financial calculations, or any kind of data that need not to be approximated. Don't worry, Python gives you theDecimal type, whichdoesn't suffer from these issues; we'll see them in a moment.

Complex numbers

Python gives youcomplex numbers support out of the box. If you don't know what complex numbers are, they arenumbers that can be expressed in theforma + ib, wherea andb are real numbers, andi (orj if you're an engineer) is the imaginary unit; that is, the square root of-1.a andb are called, respectively, thereal andimaginary part of the number.

It is perhaps unlikely that you will use them, unless you're coding something scientific. Nevertheless, let's see a small example:

>>> c =3.14 +2.73j>>> c =complex(3.14,2.73)# same as above>>> c.real# real part3.14>>> c.imag# imaginary part2.73>>> c.conjugate()# conjugate of A + Bj is A - Bj(3.14-2.73j)>>> c *2# multiplication is allowed(6.28+5.46j)>>> c **2# power operation as well(2.4067000000000007+17.1444j)>>> d =1 +1j# addition and subtraction as well>>> c - d(2.14+1.73j)

Fractions and decimals

Let's finish the tour of the number department with a look at fractions and decimals. Fractions holda rational numerator and denominator in their lowest forms. Let's see a quick example:

>>>from fractionsimport Fraction>>> Fraction(10,6)# mad hatter?Fraction(5, 3)  # notice it's been simplified>>> Fraction(1,3) + Fraction(2,3)# 1/3 + 2/3 == 3/3 == 1/1Fraction(1, 1)>>> f = Fraction(10,6)>>> f.numerator5>>> f.denominator3>>> f.as_integer_ratio()(5, 3)

Theas_integer_ratio() method has also been added to integers and Booleans. This is helpful, as it allows you to use it without needing to worry about what type of number is being worked with.

AlthoughFraction objects can be very useful at times, it's not that common to spot them in commercial software. Instead, it is much more common to see decimal numbers being used in all those contexts where precision is everything; for example, in scientific and financial calculations.

Let's see aquick examplewith decimal numbers:

>>>from decimalimport Decimalas D# rename for brevity>>> D(3.14)# pi, from float, so approximation issuesDecimal('3.140000000000000124344978758017532527446746826171875')>>> D('3.14')# pi, from a string, so no approximation issuesDecimal('3.14')>>> D(0.1) * D(3) - D(0.3)# from float, we still have the issueDecimal('2.775557561565156540423631668E-17')>>> D('0.1') * D(3) - D('0.3')# from string, all perfectDecimal('0.0')>>> D('1.4').as_integer_ratio()# 7/5 = 1.4 (isn't this cool?!)(7, 5)

Notice that when we construct aDecimal number from afloat, it takes on all the approximation issues afloat may come with. On the other hand, when we create aDecimal from an integer or a string representation of a number, then theDecimal will have no approximation issues, and therefore no quirky behavior. When it comes to currency or situations in which precision is of utmost importance, use decimals.

This concludes our introduction to built-in numeric types. Let's now look at sequences.

Let'sstart with immutable sequences: strings, tuples, and bytes.

Strings and bytes

Textual data in Pythonis handled withstr objects, more commonly known asstrings. They are immutable sequences ofUnicode code points. Unicode code points can represent a character, but can also have other meanings, such as when formatting, for example. Python, unlike other languages, doesn't have achar type, so a single character is rendered simply by a string of length 1.

Unicode is an excellent way to handle data, and should be used for the internals of any application. When it comes to storing textual data though, or sending it on the network, you will likely want to encode it, using an appropriate encoding for the medium you are using. The result of an encoding produces abytes object, whose syntax and behavior is similar to that of strings. String literals are written in Python using single, double, or triple quotes (both single or double). If built with triple quotes, a string can span multiple lines. An example will clarify this:

>>># 4 ways to make a string>>> str1 ='This is a string. We built it with single quotes.'>>> str2 ="This is also a string, but built with double quotes.">>> str3 ='''This is built using triple quotes,...so it can span multiple lines.'''>>> str4 ="""This too...is a multiline one...built with triple double-quotes.""">>> str4#A'This too\nis a multiline one\nbuilt with triple double-quotes.'>>> print(str4)#BThis toois a multiline onebuilt with triple double-quotes.

In#A and#B, we printstr4, first implicitly, and then explicitly, using theprint()function. A good exercise would be to find out why they are different. Are you up to the challenge? (Hint: look up thestr() andrepr() functions.)

Strings, like any sequence, have a length. You can get this by calling thelen() function:

>>>len(str1)49

Python 3.9 has introduced two new methods that deal with the prefixes and suffixes of strings. Here's an example that explains the way they work:

>>> s ='Hello There'>>> s.removeprefix('Hell')'o There'>>> s.removesuffix('here')'Hello T'>>> s.removeprefix('Ooops')'Hello There'

The nice thing about them is shown by the last instruction: when we attempt to remove a prefix or suffix which is not there, the method simply returns a copy of the original string. This means that these methods, behind the scenes, are checking if the prefix or suffix matches the argument of the call, and when that's the case, they remove it.

Encoding and decoding strings

Using theencode/decode methods, we canencode Unicode strings and decode bytes objects.UTF-8 is avariable-lengthcharacter encoding, capable ofencoding all possible Unicode code points. It is the most widely used encoding for the web. Notice also that by adding the literalb infront of a string declaration, we're creating abytes object:

>>> s ="This is üŋíc0de"# unicode string: code points>>>type(s)<class 'str'>>>> encoded_s = s.encode('utf-8')# utf-8 encoded version of s>>> encoded_sb'This is \xc3\xbc\xc5\x8b\xc3\xadc0de'  # result: bytes object>>>type(encoded_s)# another way to verify it<class 'bytes'>>>> encoded_s.decode('utf-8')# let's revert to the original'This is üŋíc0de'>>> bytes_obj =b"A bytes object"# a bytes object>>>type(bytes_obj)<class 'bytes'>

Indexing and slicing strings

When manipulating sequences, it's very common to access them at one precise position (indexing), or to get a sub-sequence out of them (slicing). When dealing with immutable sequences, both operations are read-only.

While indexing comes in one form—zero-based access to any position within the sequence—slicing comes in different forms. When you get a slice of a sequence, you can specify thestart andstop positions, along with thestep. They are separated with a colon (:) likethis:my_sequence[start:stop:step]. All the arguments are optional;start is inclusive, andstop is exclusive. It's probably better to see an example, rather than try to explain them any further with words:

>>> s ="The trouble is you think you have time.">>> s[0]# indexing at position 0, which is the first char'T'>>> s[5]# indexing at position 5, which is the sixth char'r'>>> s[:4]# slicing, we specify only the stop position'The '>>> s[4:]# slicing, we specify only the start position'trouble is you think you have time.'>>> s[2:14]# slicing, both start and stop positions'e trouble is'>>> s[2:14:3]# slicing, start, stop and step (every 3 chars)'erb '>>> s[:]# quick way of making a copy'The trouble is you think you have time.'

The last line is quite interesting. If you don't specify any of the parameters, Python will fill in the defaults for you. In this case,startwill be the start of the string,stopwill be the end of the string, andstepwill be the default: 1. This is an easy and quick way of obtaining a copy of the strings (the same value, but a different object). Can you think of a way to get the reversed copy of a string using slicing (don't look it up—find it for yourself)?

String formatting

One of the features strings have is theability to be used as a template. There are several different ways of formatting a string, and for the full list of possibilities, we encourage you to look up the documentation. Here are some common examples:

>>> greet_old ='Hello %s!'>>> greet_old %'Fabrizio''Hello Fabrizio!'>>> greet_positional ='Hello {}!'>>> greet_positional.format('Fabrizio')'Hello Fabrizio!'>>> greet_positional ='Hello {} {}!'>>> greet_positional.format('Fabrizio','Romano')'Hello Fabrizio Romano!'>>> greet_positional_idx ='This is {0}! {1} loves {0}!'>>> greet_positional_idx.format('Python','Heinrich')'This is Python! Heinrich loves Python!'>>> greet_positional_idx.format('Coffee','Fab')'This is Coffee! Fab loves Coffee!'>>> keyword ='Hello, my name is {name} {last_name}'>>> keyword.format(name='Fabrizio', last_name='Romano')'Hello, my name is Fabrizio Romano'

In the previous example, you can see four different ways of formatting strings. The first one, which relies on the% operator, is deprecated and shouldn't be used anymore. The current, modern way to format a string is by using theformat() string method. You can see, from the different examples, that a pair of curly braces acts as a placeholder within the string. When we callformat(), we feed it data that replaces the placeholders. We can specify indexes (and much more) within the curly braces, and even names, which implies we'll have to callformat() using keyword arguments instead of positional ones.

Notice howgreet_positional_idx is rendered differently by feeding different data to the call toformat.

One last feature we want to show you was added to Python in version 3.6, and it's calledformatted string literals. This feature isquite cool (and it is faster than using theformat() method): strings are prefixed withf, and contain replacement fields surrounded by curly braces.

Replacement fields are expressions evaluated at runtime, and then formatted using the format protocol:

>>> name ='Fab'>>> age =42>>>f"Hello! My name is{name} and I'm{age}""Hello! My name is Fab and I'm 42">>>from mathimport pi>>>f"No arguing with{pi}, it's irrational...""No arguing with 3.141592653589793, it's irrational..."

An interesting addition to f-strings, which wasintroduced in Python 3.8, is the ability to add an equals sign specifier within the f-string clause; this causes the expression to expand to the text of the expression, an equals sign, then the representation of the evaluated expression. This is great for self-documenting and debugging purposes. Here's an example that shows the difference in behavior:

>>> user ='heinrich'>>> password ='super-secret'>>>f"Log in with:{user} and{password}"'Log in with: heinrich and super-secret'>>>f"Log in with:{user=} and{password=}""Log in with: user='heinrich' and password='super-secret'"

Check out the official documentation to learn everything about string formatting and how truly powerful it can be.

Tuples

The last immutable sequence type we are going to look at here is thetuple. A tuple is a sequence of arbitrary Python objects. In a tuple declaration, items are separated by commas. Tuples are used everywhere in Python. They allow for patterns that are quite hard to reproduce in other languages. Sometimes tuples are used implicitly; for example, to set up multiple variables on one line, or to allow a function to return multiple objects (in several languages, it is common for a function to return only one object), and in the Python console, tuples can be used implicitly to print multiple elements with one single instruction. We'll see examples for all these cases:

>>> t = ()# empty tuple>>>type(t)<class 'tuple'>>>> one_element_tuple = (42, )# you need the comma!>>> three_elements_tuple = (1,3,5)# braces are optional here>>> a, b, c =1,2,3# tuple for multiple assignment>>> a, b, c# implicit tuple to print with one instruction(1, 2, 3)>>>3in three_elements_tuple# membership testTrue

Notice that the membership operatorin can also be used with lists, strings, dictionaries, and, in general, with collection and sequence objects.

One thing that tuple assignmentallows us to do isone-line swaps, with no need for a third temporary variable. Let's first see the traditional way of doing it:

>>> a, b =1,2>>> c = a# we need three lines and a temporary var c>>> a = b>>> b = c>>> a, b# a and b have been swapped(2, 1)Now let's see how we would do it in Python:>>> a, b =0,1>>> a, b = b, a# this is the Pythonic way to do it>>> a, b(1, 0)

Take a look at the line that shows you the Pythonic way of swapping two values. Do you remember what we wrote inChapter 1,A Gentle Introduction to Python? A Python program is typically one-fifth to one-third the size of equivalent Java or C++ code, and features like one-line swaps contribute to this. Python is elegant, where elegance in this context also means economy.

Because they are immutable, tuplescan be used as keys for dictionaries (we'll see this shortly). To us, tuples are Python's built-in data that most closely represent a mathematical vector. This doesn't mean that this was the reason for which they were created, though. Tuples usually contain a heterogeneous sequence of elements while, on the other hand, lists are, most of the time, homogeneous. Moreover, tuples are normally accessed via unpacking or indexing, while lists are usually iterated over.

Mutable sequences differ fromtheir immutable counterparts in that they can be changed after creation. There are two mutable sequence types in Python:lists andbyte arrays.

Lists

Python lists are very similar to tuples, but they don't have the restrictions of immutability. Lists are commonly used for storing collections of homogeneous objects, but there is nothing preventing you from storing heterogeneous collections as well. Lists can be created in many different ways. Let's see an example:

>>> []# empty list[]>>>list()# same as [][]>>> [1,2,3]# as with tuples, items are comma separated[1, 2, 3]>>> [x +5for xin [2,3,4]]# Python is magic[7, 8, 9]>>>list((1,3,5,7,9))# list from a tuple[1, 3, 5, 7, 9]>>>list('hello')# list from a string['h', 'e', 'l', 'l', 'o']

In the previous example, we showed you how to create a list using various techniques. We would like you to take a good look at the line with the commentPython is magic, which we don't expect you to fully understand at this point—especially if you are unfamiliar with Python. That is called alist comprehension: a very powerful functional feature of Python, which we will see in detail inChapter 5,Comprehensions and Generators. We just wanted to spark your curiosity at this point.

Creating lists is good, but the real fun beginswhen we use them, so let's see the main methods they gift us with:

>>> a = [1,2,1,3]>>> a.append(13)# we can append anything at the end>>> a[1, 2, 1, 3, 13]>>> a.count(1)# how many `1s` are there in the list?2>>> a.extend([5,7])# extend the list by another (or sequence)>>> a[1, 2, 1, 3, 13, 5, 7]>>> a.index(13)# position of `13` in the list (0-based indexing)4>>> a.insert(0,17)# insert `17` at position 0>>> a[17, 1, 2, 1, 3, 13, 5, 7]>>> a.pop()# pop (remove and return) last element7>>> a.pop(3)# pop element at position 31>>> a[17, 1, 2, 3, 13, 5]>>> a.remove(17)# remove `17` from the list>>> a[1, 2, 3, 13, 5]>>> a.reverse()# reverse the order of the elements in the list>>> a[5, 13, 3, 2, 1]>>> a.sort()# sort the list>>> a[1, 2, 3, 5, 13]>>> a.clear()# remove all elements from the list>>> a[]

The preceding code gives you aroundup of a list's main methods. We want to show you how powerful they are, using the methodextend() as an example. You can extend lists using any sequence type:

>>> a =list('hello')# makes a list from a string>>> a['h', 'e', 'l', 'l', 'o']>>> a.append(100)# append 100, heterogeneous type>>> a['h', 'e', 'l', 'l', 'o', 100]>>> a.extend((1,2,3))# extend using tuple>>> a['h', 'e', 'l', 'l', 'o', 100, 1, 2, 3]>>> a.extend('...')# extend using string>>> a['h', 'e', 'l', 'l', 'o', 100, 1, 2, 3, '.', '.', '.']

Now, let's see the most common operations you can do with lists:

>>> a = [1,3,5,7]>>>min(a)# minimum value in the list1>>>max(a)# maximum value in the list7>>>sum(a)# sum of all values in the list16>>>from mathimport prod>>> prod(a)# product of all values in the list105>>>len(a)# number of elements in the list4>>> b = [6,7,8]>>> a + b# `+` with list means concatenation[1, 3, 5, 7, 6, 7, 8]>>> a *2# `*` has also a special meaning[1, 3, 5, 7, 1, 3, 5, 7]

Notice how easily we can perform the sum and the product of all values in a list. The functionprod(), from themath module, is just one of the many new additions introduced in Python 3.8. Even if you don't plan to use it that often, it's always a good idea to check out themath module and be familiar with its functions, as they can be quite helpful.

The last two lines in the preceding code are also quite interesting, as they introduce us to a conceptcalledoperator overloading. In short, this means that operators, such as+,-,*,%, and so on, may represent different operations according to the context they are used in. It doesn't make any sense to sum two lists, right? Therefore, the+ sign is used to concatenate them. Hence, the* sign is used to concatenate the list to itself according to the right operand.

Now, let's take a step further and see something a little more interesting. We want to show you how powerful thesorted method can be and how easy it is in Python to achieve results that require a great deal of effort in other languages:

>>>from operatorimport itemgetter>>> a = [(5,3), (1,3), (1,2), (2, -1), (4,9)]>>>sorted(a)[(1, 2), (1, 3), (2, -1), (4, 9), (5, 3)]>>>sorted(a, key=itemgetter(0))[(1, 3), (1, 2), (2, -1), (4, 9), (5, 3)]>>>sorted(a, key=itemgetter(0,1))[(1, 2), (1, 3), (2, -1), (4, 9), (5, 3)]>>>sorted(a, key=itemgetter(1))[(2, -1), (1, 2), (5, 3), (1, 3), (4, 9)]>>>sorted(a, key=itemgetter(1), reverse=True)[(4, 9), (5, 3), (1, 3), (1, 2), (2, -1)]

The preceding code deserves a little explanation. First of all,a is a list of tuples. This means each element ina is a tuple (a 2-tuple in this case). When we callsorted(my_list), we get a sorted version ofmy_list. In this case, the sorting on a 2-tuple works by sorting them on the first item in the tuple, and on the second when the first one is the same. You can see this behavior in the result ofsorted(a), which yields[(1, 2), (1, 3), ...]. Python also gives us the ability to control which element(s) of the tuple the sorting must be run against. Notice that when we instruct thesorted function, to work on the first element of each tuple (withkey=itemgetter(0)), the result is different:[(1, 3), (1, 2), ...]. The sorting is done only on the first element of each tuple (which is the one at position 0). If we want to replicate the default behavior of a simplesorted(a) call, we need to usekey=itemgetter(0, 1), which tells Python to sort first on the elements at position 0 within the tuples, and then on those at position 1. Compare the results and you will see that they match.

For completeness, we included an example of sorting only on the elements at position 1, and then again, with the same sorting but in reverse order. If you have ever seen sorting in other languages, you should be quite impressed at this moment.

The Python sorting algorithm is very powerful, and it was written by Tim Peters (we've already seen this name, can you recall when?). It is aptly namedTimsort, and it is a blend betweenmerge andinsertion sort and has better time performances than most otheralgorithms used for mainstream programming languages. Timsort is a stable sorting algorithm, which means that when multiple records score the same in the comparison, their original order is preserved. We've seen this in the result ofsorted(a, key=itemgetter(0)), which yielded[(1, 3), (1, 2), ...], in which the order of those two tuples had been preserved because they had the same value at position 0.

Bytearrays

To conclude our overview ofmutable sequence types, let's spend a moment on thebytearray type. Basically, they represent the mutable version of bytes objects. They expose most of the usual methods of mutable sequences as well as most of the methods of the bytes type. Items in a bytearray are integers in the range [0, 256).

Let's see an example with thebytearray type:

>>>bytearray()# empty bytearray objectbytearray(b'')>>>bytearray(10)# zero-filled instance with given lengthbytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')>>>bytearray(range(5))# bytearray from iterable of integersbytearray(b'\x00\x01\x02\x03\x04')>>> name =bytearray(b'Lina')#A - bytearray from bytes>>> name.replace(b'L',b'l')bytearray(b'lina')>>> name.endswith(b'na')True>>> name.upper()bytearray(b'LINA')>>> name.count(b'L')1

As you can see, there are a few ways to create abytearray object. They can be useful inmany situations; for example, when receiving data through a socket, they eliminate the need to concatenate data while polling, hence they can prove to be very handy. On line#A, we created abytearray named asname from the bytes literalb'Lina' to show you how thebytearray object exposes methods from both sequences and strings, which is extremely handy. If you think about it, they can be considered as mutable strings.

Python also provides two set types,set andfrozenset. Theset type is mutable, whilefrozenset is immutable. They are unordered collections of immutable objects.Hashability is a characteristic that allows an object to be used as a set member as well as a key for a dictionary, as we'll see very soon.

Objects that compare equally must have the same hash value. Setsare very commonly used to test for membership; let's introduce thein operator in the following example:

>>> small_primes =set()# empty set>>> small_primes.add(2)# adding one element at a time>>> small_primes.add(3)>>> small_primes.add(5)>>> small_primes{2, 3, 5}>>> small_primes.add(1)# Look what I've done, 1 is not a prime!>>> small_primes{1, 2, 3, 5}>>> small_primes.remove(1)# so let's remove it>>>3in small_primes# membership testTrue>>>4in small_primesFalse>>>4notin small_primes# negated membership testTrue>>> small_primes.add(3)# trying to add 3 again>>> small_primes{2, 3, 5}  # no change, duplication is not allowed>>> bigger_primes =set([5,7,11,13])# faster creation>>> small_primes | bigger_primes# union operator `|`{2, 3, 5, 7, 11, 13}>>> small_primes & bigger_primes# intersection operator `&`{5}>>> small_primes - bigger_primes# difference operator `-`{2, 3}

In the preceding code, you can see two different ways to create a set. One creates an empty set and then adds elements one at a time. The other creates the set using a list of numbers as an argument to the constructor, which does all the work for us. Of course, you can create a set from a list or tuple (or any iterable) and then you can add and remove members from the set as you please.

Another way of creating a set is by simply using the curly braces notation, like this:

>>> small_primes = {2,3,5,5,3}>>> small_primes{2, 3, 5}

Notice we added some duplication to emphasize that the resulting set won't have any. Let's see an example using the immutable counterpart of the set type,frozenset:

>>> small_primes =frozenset([2,3,5,7])>>> bigger_primes =frozenset([5,7,11])>>> small_primes.add(11)# we cannot add to a frozensetTraceback (most recent call last):  File "<stdin>", line 1, in <module>AttributeError: 'frozenset' object has no attribute 'add'>>> small_primes.remove(2)# nor can we removeTraceback (most recent call last):  File "<stdin>", line 1, in <module>AttributeError: 'frozenset' object has no attribute 'remove'>>> small_primes & bigger_primes# intersect, union, etc. allowedfrozenset({5, 7})

As you can see,frozenset objectsare quite limited with respect to their mutable counterpart. They still prove very effective for membership test, union, intersection, and difference operations, and for performance reasons.

Of all the built-in Python data types, the dictionary is easily the most interesting. It's the only standard mapping type, and it is thebackbone of every Python object.

A dictionary maps keys to values. Keys need to be hashable objects, while values can be of any arbitrary type. Dictionaries are also mutable objects. There are quite a few different ways to create a dictionary, so let us give you a simple example of how to create a dictionary equal to{'A': 1, 'Z': -1} in five different ways:

>>> a =dict(A=1, Z=-1)>>> b = {'A':1,'Z': -1}>>> c =dict(zip(['A','Z'], [1, -1]))>>> d =dict([('A',1), ('Z', -1)])>>> e =dict({'Z': -1,'A':1})>>> a == b == c == d == e# are they all the same?True  # They are indeed

Have you noticed those double equals? Assignment is done with one equal, while to check whether an object is the same as another one (or five in one go, in this case), we use double equals. There is also another way to compare objects, which involves theis operator, and checks whether the two objects are the same (that is, that they have the same ID, not just the same value), but unless you have a good reason to use it, you should use the double equals instead. In the preceding code, we also used one nice function:zip(). It is named after the real-life zip, which glues together two parts, taking one element from each part at a time. Let us show you an example:

>>>list(zip(['h','e','l','l','o'], [1,2,3,4,5]))[('h', 1), ('e', 2), ('l', 3), ('l', 4), ('o', 5)]>>>list(zip('hello',range(1,6)))# equivalent, more pythonic[('h', 1), ('e', 2), ('l', 3), ('l', 4), ('o', 5)]

In the preceding example, we have created the same list in two different ways, one more explicit, and the other a little bit more Pythonic. Forget for a moment that we had to wrap thelist() constructor around thezip() call (the reason iszip() returns an iterator, not alist, so if we want to see the result, we need to exhaust that iterator into something—a list in this case), and concentrate on the result. See howzip() has coupled the first elements of its two arguments together, then the second ones, then the third ones, and so on?

Take a look at the zip of your suitcase, or a purse, or the cover of a pillow, and you will see it works exactly like the one in Python. But let's go back to dictionaries and see how many wonderful methods they expose for allowing us to manipulate them as we want. Let's start with the basic operations:

>>> d = {}>>> d['a'] =1# let's set a couple of (key, value) pairs>>> d['b'] =2>>>len(d)# how many pairs?2>>> d['a']# what is the value of 'a'?1>>> d# how does `d` look now?{'a': 1, 'b': 2}>>>del d['a']# let's remove `a`>>> d{'b': 2}>>> d['c'] =3# let's add 'c': 3>>>'c'in d# membership is checked against the keysTrue>>>3in d# not the valuesFalse>>>'e'in dFalse>>> d.clear()# let's clean everything from this dictionary>>> d{}

Notice howaccessing keys of a dictionary, regardless of the type of operation we're performing, is done using square brackets. Do you remember strings, lists, and tuples? We were accessing elements at some position through square brackets as well, which is yet anotherexample of Python's consistency.

Let's now look at three special objects calleddictionary views:keys,values, anditems. These objects provide a dynamic view of the dictionary entries and they change when the dictionary changes.keys() returns all the keys in the dictionary,values() returns all the values in the dictionary, anditems() returns all the(key, value) pairs in thedictionary.

Enough with this chatter; let's put all this down into code:

>>> d =dict(zip('hello',range(5)))>>> d{'h': 0, 'e': 1, 'l': 3, 'o': 4}>>> d.keys()dict_keys(['h', 'e', 'l', 'o'])>>> d.values()dict_values([0, 1, 3, 4])>>> d.items()dict_items([('h', 0), ('e', 1), ('l', 3), ('o', 4)])>>>3in d.values()True>>> ('o',4)in d.items()True

There are a few things to note here. First, notice how we are creating a dictionary by iterating over the zipped version of the string'hello' and the list[0, 1, 2, 3, 4]. The string'hello' has two'l' characters inside, and they are paired up with the values 2 and 3 by thezip() function. Notice how in the dictionary, the second occurrence of the'l' key (the one with the value 3), overwrites the first one (the one with the value 2). Another thing to notice is that when asking for any view, the original order in which items were added is now preserved, while before version 3.6 there was no guarantee of that.

As of Python 3.6, thedict type has been reimplemented to use a more compact representation. This resulted in dictionaries using 20% to 25% less memory when comparedto Python 3.5. Moreover, since Python 3.6, as a side effect, dictionaries preserve the order in which keys were inserted. This feature has received such a welcome from the community that in 3.7 it has become an official feature of the language rather than an implementation side effect. Since Python 3.8, dictionaries are also reversible!

We'll see how these views are fundamental tools when we talk about iterating over collections. Let's take a look now at some other methods exposed by Python's dictionaries—there's plenty of them and they are very useful:

>>> d{'h': 0, 'e': 1, 'l': 3, 'o': 4}>>> d.popitem()# removes a random item (useful in algorithms)('o', 4)>>> d{'h': 0, 'e': 1, 'l': 3}>>> d.pop('l')# remove item with key `l`3>>> d.pop('not-a-key')# remove a key not in dictionary: KeyErrorTraceback (most recent call last):  File "<stdin>", line 1, in <module>KeyError: 'not-a-key'>>> d.pop('not-a-key','default-value')# with a default value?'default-value'# we get the default value>>> d.update({'another':'value'})# we can update dict this way>>> d.update(a=13)# or this way (like a function call)>>> d{'h': 0, 'e': 1, 'another': 'value', 'a': 13}>>> d.get('a')# same as d['a'] but if key is missing no KeyError13>>> d.get('a',177)# default value used if key is missing13>>> d.get('b',177)# like in this case177>>> d.get('b')# key is not there, so None is returned

All these methods are quite simple to understand, but it's worth talking about thatNone, for a moment. Every function in Python returnsNone, unless thereturn statement is explicitly used to return something else, but we'll see this when we explore functions.None is frequently used to represent the absence of a value, and it is quite commonly used as a default value for arguments in function declaration. Some inexperienced coders sometimes write code that returns eitherFalse orNone. BothFalse andNone evaluate toFalse in a Boolean context, so it may seem that there is not much difference between them. But actually, we would argue the contrary, that there is an important difference:False means that we have information, and the information we have isFalse.None meansno information; no information is very different from information that isFalse. In layman's terms, if you ask your mechanicIs my car ready?, there is a big difference between the answerNo, it's not (False) andI have no idea (None).

One last method we really like about dictionaries issetdefault(). It behaves likeget(), but also sets the keywith the given value if it is not there. Let's see an example:

>>> d = {}>>> d.setdefault('a',1)# 'a' is missing, we get default value1>>> d{'a': 1}# also, the key/value pair ('a', 1) has now been added>>> d.setdefault('a',5)# let's try to override the value1>>> d{'a': 1} # no override, as expected

This brings us to the end of this tour of dictionaries. Test your knowledge about them by trying to foresee whatd looks like after this line:

>>> d = {}>>> d.setdefault('a', {}).setdefault('b', []).append(1)

Don't worry if you don't get it immediately. We just want to encourage you to experiment with dictionaries.

Python 3.9 sports a brand-new union operator available fordict objects, which was introduced byPEP 584. When itcomes to applying union todict objects, we need to remember that union for them is not commutative. This becomes evident when the twodict objects we're merging have one or more keys in common. Check out this example:

>>> d = {'a':'A','b':'B'}>>> e = {'b':8,'c':'C'}>>> d | e{'a': 'A', 'b': 8, 'c': 'C'}>>> e | d{'b': 'B', 'c': 'C', 'a': 'A'}>>> {**d, **e}{'a': 'A', 'b': 8, 'c': 'C'}>>> {**e, **d}{'b': 'B', 'c': 'C', 'a': 'A'}>>> d |= e>>> d{'a': 'A', 'b': 8, 'c': 'C'}

Here,dict objectsd ande have the key'b' in common. For thedict object,d, the value associated with'b' is'B'; whereas, fordicte, it's the number 8. This means that when we merge them withe on the righthand side of the union operator,|, the value ine overrides the one ind. The opposite happens, of course, when we swap the positions of those objects in relation to the union operator.

In this example, you can also see how the union can be performed by using the** operator to produceadictionary unpacking. It's worth noting that union can also be performed as an augmented assignment operation (d |= e), which works in place. Please refer to PEP 584 for more information about this feature.

This concludes our tour of built-in data types. Before we discuss some considerations about what we've seen in this chapter, we briefly want to take a peek at data types.

Python provides a variety ofspecialized data types, such as dates and times, container types, and enumerations. There is a whole section in the Python standard library titledData Types, which deserves to be explored; it is filled with interesting and useful tools for each andevery programmer's needs. You can find it here:

https://docs.python.org/3/library/datatypes.html

In this section, we are briefly going to take a look at dates and times, collections, and enumerations.

Dates and times

The Python standard library provides several data types that can be used to deal with dates and times. This realm may seem innocuous at first glance, but it's actually quite tricky: timezones, daylight saving time… There are a huge number of ways to format date and time information; calendar quirks, parsing, and localizing—these are just a few of the many difficulties we face when we deal with dates and times, and that's probably the reason why, in this particular context, it is very common for professional Python programmers to also rely on various third-party libraries that provide some much-needed extra power.

The standard library

We will start with the standard library, and finish the session with a little overview of what's out there in terms of the third-party libraries you can use.

From the standardlibrary, themain modules that are used to handle dates and times aredatetime,calendar,zoneinfo, andtime. Let's start with the imports you'llneed forthis whole section:

>>>from datetimeimport date, datetime, timedelta, timezone>>>import time>>>import calendaras cal>>>from zoneinfoimport ZoneInfo

The first example deals with dates. Let's see how they look:

>>> today = date.today()>>> todaydatetime.date(2021, 3, 28)>>> today.ctime()'Sun Mar 28 00:00:00 2021'>>> today.isoformat()'2021-03-28'>>> today.weekday()6>>> cal.day_name[today.weekday()]'Sunday'>>> today.day, today.month, today.year(28, 3, 2021)>>> today.timetuple()time.struct_time(    tm_year=2021, tm_mon=3, tm_mday=28,    tm_hour=0, tm_min=0, tm_sec=0,    tm_wday=6, tm_yday=87, tm_isdst=-1)

We start by fetching the date for today. We can see that it's an instance of thedatetime.date class. Then we get two different representations for it, following theC and theISO 8601 format standards, respectively. After that, we ask what day of the week it is, and we get the number 6. Days are numbered 0 to 6 (representing Monday to Sunday), so we grab the value of the sixth element incalendar.day_name (notice in the code that we have substitutedcalendar with "cal" for brevity).

The last two instructions show how to get detailed information out of a date object. We can inspect itsday,month, andyear attributes, or call thetimetuple() method and get a whole wealth of information. Since we're dealing with a date object, notice that all the information about time has been set to 0.

Let's now play with time:

>>> time.ctime()'Sun Mar 28 15:23:17 2021'>>> time.daylight1>>> time.gmtime()time.struct_time(    tm_year=2021, tm_mon=3, tm_mday=28,    tm_hour=14, tm_min=23, tm_sec=34,    tm_wday=6, tm_yday=87, tm_isdst=0)>>> time.gmtime(0)time.struct_time(    tm_year=1970, tm_mon=1, tm_mday=1,    tm_hour=0, tm_min=0, tm_sec=0,    tm_wday=3, tm_yday=1, tm_isdst=0)>>> time.localtime()time.struct_time(    tm_year=2021, tm_mon=3, tm_mday=28,    tm_hour=15, tm_min=23, tm_sec=50,    tm_wday=6, tm_yday=87, tm_isdst=1)>>> time.time()1616941458.149149

This example is quitesimilar to the one before, only here, we are dealing with time. We can see how to get a printed representation of time according to C format standard, and then how to check if daylight saving time is in effect. The functiongmtime converts a given number of seconds from the epoch to astruct_time object in UTC. If we don't feed it any number, it will use the current time.

We finish the example by getting thestruct_time object for the current local time and the number of seconds from the epoch expressed as a float number (time.time()).

Let's now see an example usingdatetime objects, which bring together dates and times.

>>> now = datetime.now()>>> utcnow = datetime.utcnow()>>> nowdatetime.datetime(2021, 3, 28, 15, 25, 16, 258274)>>> utcnowdatetime.datetime(2021, 3, 28, 14, 25, 22, 918195)>>> now.date()datetime.date(2021, 3, 28)>>> now.day, now.month, now.year(28, 3, 2021)>>> now.date() == date.today()True>>> now.time()datetime.time(15, 25, 16, 258274)>>> now.hour, now.minute, now.second, now.microsecond(15, 25, 16, 258274)>>> now.ctime()'Sun Mar 28 15:25:16 2021'>>> now.isoformat()'2021-03-28T15:25:16.258274'>>> now.timetuple()time.struct_time(    tm_year=2021, tm_mon=3, tm_mday=28,    tm_hour=15, tm_min=25, tm_sec=16,    tm_wday=6, tm_yday=87, tm_isdst=-1)>>> now.tzinfo>>> utcnow.tzinfo>>> now.weekday()6

The preceding example is rather self-explanatory. We start by setting up two instances that represent the current time. One is related to UTC (utcnow), and the other one is a local representation (now). It just so happens that we ran this code on the first day after daylight saving time was introduced in the UK in 2021, sonow represents the current time in BST. BST is one hour ahead of UTC when daylight saving time is in effect, as can be seen from the code.

You can getdate,time, and specific attributes from adatetime object in a similar way as to what we have already seen. It is also worth noting how bothnow andutcnow present the valueNone for thetzinfo attribute. This happens because those objects arenaive.

Let's now see how a duration is represented in this context:

>>> f_bday = datetime(    1975, 12, 29, 12, 50, tzinfo=ZoneInfo('Europe/Rome')    )>>> h_bday = datetime(    1981, 10, 7, 15, 30, 50, tzinfo=timezone(timedelta(hours=2))    )>>> diff = h_bday - f_bday>>>type(diff)<class 'datetime.timedelta'>>>> diff.days2109>>> diff.total_seconds()182223650.0>>> today + timedelta(days=49)datetime.date(2021, 5, 16)>>> now + timedelta(weeks=7)datetime.datetime(2021, 5, 16, 15, 25, 16, 258274)

Two objects have been created that represent Fabrizio and Heinrich's birthdays. This time, in order to show you the alternative, we have createdaware objects.

There are severalways to include time zone information when creating adatetime object, and in this example, we are showing you two of them. One uses the brand-newZoneInfo object from thezoneinfo module, introduced in Python 3.9. The second one uses a simpletimedelta, an object that represents a duration.

We then create thediff object, which is assigned as the subtraction of them. The result of that operation is an instance oftimedelta. You can see how we can interrogate thediff object to tell us how many days Fabrizio and Heinrich's birthdays are apart, and even the number of seconds that represent that whole duration. Notice that we need to usetotal_seconds, which expresses the whole duration in seconds. Theseconds attribute represents the number of seconds assigned to that duration. So, atimedelta(days=1) will have seconds equal to 0, andtotal_seconds equal to 86,400 (which is the number of seconds in a day).

Combining adatetime with a duration adds or subtracts that duration from the original date and time information. In the last few lines of the example, we can see how adding a duration to adate object produces adate as a result, whereas adding it to adatetime produces adatetime, as it is fair to expect.

One of the more difficult undertakings to carry out using dates and times is parsing. Let's see a short example:

>>> datetime.fromisoformat('1977-11-24T19:30:13+01:00')datetime.datetime(    1977, 11, 24, 19, 30, 13,    tzinfo=datetime.timezone(datetime.timedelta(seconds=3600)))>>> datetime.fromtimestamp(time.time())datetime.datetime(2021, 3, 28, 15, 42, 2, 142696)

We can easily createdatetime objects from ISO-formatted strings, as well as from timestamps. However, in general, parsing a date from unknown formats can prove to be a difficult task.

Third-party libraries

To finish off this subsection, wewould like to mention a few third-party libraries that you will very likely come across the moment you will have to deal with dates and times in your code:

• dateutil: Powerfulextensions todatetime (https://dateutil.readthedocs.io/en/stable/)
• Arrow: Betterdates and times for Python (https://arrow.readthedocs.io/en/latest/)
• pytz: Worldtime zone definitions for Python (https://pythonhosted.org/pytz/)

These three are some of the most common, and they are worth investigating.

Let's take a look at one final example, this time using the Arrow third-party library:

>>>import arrow>>> arrow.utcnow()<Arrow [2021-03-28T14:43:20.017213+00:00]>>>> arrow.now()<Arrow [2021-03-28T15:43:39.370099+01:00]>>>> local = arrow.now('Europe/Rome')>>> local<Arrow [2021-03-28T16:59:14.093960+02:00]>>>> local.to('utc')<Arrow [2021-03-28T14:59:14.093960+00:00]>>>> local.to('Europe/Moscow')<Arrow [2021-03-28T17:59:14.093960+03:00]>>>> local.to('Asia/Tokyo')<Arrow [2021-03-28T23:59:14.093960+09:00]>>>> local.datetimedatetime.datetime(    2021, 3, 28, 16, 59, 14, 93960,    tzinfo=tzfile('/usr/share/zoneinfo/Europe/Rome'))>>> local.isoformat()'2021-03-28T16:59:14.093960+02:00'

Arrow provides a wrapper around the data structures of the standard library, plus a whole set of methods and helpers that simplify the task of dealing with dates and times. You can see from this example how easy it is to get the local date and time in the Italian timezone (Europe/Rome), as well as to convert it to UTC, or to the Russian or Japanese time zones. The last two instructions show how you can get the underlyingdatetime object from an Arrow one, and the very useful ISO-formatted representation of a date and time.

The collections module

When Python general-purpose built-in containers (tuple,list,set, anddict) aren't enough, wecan find specialized container data types in thecollections module. They are described inTable 2.1.

There isn't enough space here to cover them all, but you can find plenty of examples in the official documentation; here, we will just give a small example to show younamedtuple,defaultdict, andChainMap.

namedtuple

Anamedtuple is a tuple-like object that has fields accessible by attribute lookup, as well as being indexable and iterable (it's actually a subclass oftuple). This is sort of a compromise between a fully-fledged object and a tuple, and it can be useful in those cases where you don't need the full power of a custom object, but only want your code to be more readable by avoiding weird indexing. Another use case is when there is a chance that items in the tuple need to change their position after refactoring, forcing the coder to also refactor all the logic involved, which can be very tricky.

For example, say we are handling data about the left and right eyes of a patient. We save one value for the left eye (position 0) and one for the right eye (position 1) in a regular tuple. Here's how that may look:

>>> vision = (9.5,8.8)>>> vision(9.5, 8.8)>>> vision[0]# left eye (implicit positional reference)9.5>>> vision[1]# right eye (implicit positional reference)8.8

Now let's pretend we handlevision objects all of the time, and, at some point, the designer decides to enhance them by adding information for the combined vision, so that avision object stores data in this format(left eye, combined, right eye).

Do you see the trouble we're in now? We may have a lot of code that depends onvision[0] being the left eye information (which it still is) andvision[1] being the right eye information (which is no longer the case). We have to refactor our code wherever we handle these objects, changingvision[1] tovision[2], and that can be painful. We could have probably approached this a bit better from the beginning, by using anamedtuple. Let us show you what we mean:

>>>from collectionsimport namedtuple>>> Vision = namedtuple('Vision', ['left','right'])>>> vision = Vision(9.5,8.8)>>> vision[0]9.5>>> vision.left# same as vision[0], but explicit9.5>>> vision.right# same as vision[1], but explicit8.8

If, within our code, we refer to the left and right eyes usingvision.left andvision.right, all we need to do to fix the new design issue is change our factory and the way we create instances—the rest of the code won't need to change:

>>> Vision = namedtuple('Vision', ['left','combined','right'])>>> vision = Vision(9.5,9.2,8.8)>>> vision.left# still correct9.5>>> vision.right# still correct (though now is vision[2])8.8>>> vision.combined# the new vision[1]9.2

You can see how convenient it is to refer to those values by name rather than by position. After all, as a wise man once wrote,Explicit is better than implicit (Can you recall where? ThinkZen if you can't...). This example may be a little extreme; of course, it's not likely that our code designer will go for a change like this, but you'd be amazed to see how frequently issues similar to this one occur in a professional environment, and how painful it is to refactor in such cases.

defaultdict

Thedefaultdict data typeis one of our favorites. It allows you to avoid checking whether akey is in a dictionary by simply inserting it for you on your first access attempt, with a default value whose type you pass on creation. In some cases, this tool can be very handy and shorten your code a little. Let's see a quick example. Say we are updating the value ofage, by adding one year. Ifage is not there, we assume it was 0 and we update it to 1:

>>> d = {}>>> d['age'] = d.get('age',0) +1# age not there, we get 0 + 1>>> d{'age': 1}>>> d = {'age':39}>>> d['age'] = d.get('age',0) +1# age is there, we get 40>>> d{'age': 40}

Now let's see how it would work with adefaultdict data type. The second line is actually the short version of anif clause that runs to a length of four lines, and that we would have to write if dictionaries didn't have theget() method (we'll see all aboutif clauses inChapter 3,Conditionals and Iteration):

>>>from collectionsimport defaultdict>>> dd = defaultdict(int)# int is the default type (0 the value)>>> dd['age'] +=1# short for dd['age'] = dd['age'] + 1>>> dddefaultdict(<class 'int'>, {'age': 1})  # 1, as expected

Notice how we just need to instruct thedefaultdict factory that we want anint number to be used if the key is missing (we'll get 0, which is the default for theint type). Also notice that even though in this example there is no gain on the number of lines, there is definitely a gain in readability, which is very important. You can also use a different technique to instantiate adefaultdict data type, which involves creating a factory object. To dig deeper, please refer to the official documentation.

ChainMap

ChainMap is anextremelyuseful data type which was introduced in Python 3.3. It behaves like a normal dictionary but, according to the Python documentation,is provided for quickly linking a number of mappings so they can be treated as a single unit. This is usually much faster than creating one dictionary and running multipleupdate calls on it.ChainMap can be used to simulate nested scopes and is useful in templating. The underlying mappings are stored in a list. That list is public and can be accessed or updated using themaps attribute. Lookups search the underlying mappings successively until a key is found. By contrast, writes, updates, and deletions only operate on the first mapping.

A very common use case is providing defaults, so let's see an example:

>>>from collectionsimport ChainMap>>> default_connection = {'host':'localhost','port':4567}>>> connection = {'port':5678}>>> conn = ChainMap(connection, default_connection)# map creation>>> conn['port']# port is found in the first dictionary5678>>> conn['host']# host is fetched from the second dictionary'localhost'>>> conn.maps# we can see the mapping objects[{'port': 5678}, {'host': 'localhost', 'port': 4567}]>>> conn['host'] ='packtpub.com'# let's add host>>> conn.maps[{'port': 5678, 'host': 'packtpub.com'}, {'host': 'localhost', 'port': 4567}]>>>del conn['port']# let's remove the port information>>> conn.maps[{'host': 'packtpub.com'}, {'host': 'localhost', 'port': 4567}]>>> conn['port']# now port is fetched from the second dictionary4567>>>dict(conn)# easy to merge and convert to regular dictionary{'host': 'packtpub.com', 'port': 4567}

Isn't it just lovely that Python makes your life so easy? You work on aChainMap object, configure the first mapping as you want, and when you need a complete dictionary with all the defaults as well as the customized items, you can just feed theChainMap object to adict constructor. Ifyou haveever coded in other languages, such as Java or C++, you probably will be able to appreciate how precious this is, and how well Python simplifies some tasks.

Enums

Technically not a built-in data type, as you have to import them fromtheenum module, but definitely worth mentioning, areenumerations. They were introduced in Python 3.4, and though it is not that common to see them in professional code, we thought it would be a good idea to give you an example anyway for the sake of completeness.

The official definition of an enumeration is that it is a set of symbolic names (members) bound to unique, constant values. Within an enumeration, the members can be compared by identity, and the enumeration itself can be iterated over.

Say you need to represent traffic lights; in your code, you might resort to the following:

>>> GREEN =1>>> YELLOW =2>>> RED =4>>> TRAFFIC_LIGHTS = (GREEN, YELLOW, RED)>>># or with a dict>>> traffic_lights = {'GREEN':1,'YELLOW':2,'RED':4}

There's nothing special about this code. It's something, in fact, that is very common to find. But, consider doing this instead:

>>>from enumimport Enum>>>classTrafficLight(Enum):...     GREEN =1...     YELLOW =2...     RED =4...>>> TrafficLight.GREEN<TrafficLight.GREEN: 1>>>> TrafficLight.GREEN.name'GREEN'>>> TrafficLight.GREEN.value1>>> TrafficLight(1)<TrafficLight.GREEN: 1>>>> TrafficLight(4)<TrafficLight.RED: 4>

Ignoring for a moment the (relative) complexity of a class definition, you can appreciate how this approach may be advantageous. The data structure is much cleaner, and the API it provides is much more powerful. Weencourage you to check out the official documentation to explore all the great features you can find in theenum module. We think it's worth exploring, at least once.

That's it. Now you have seen a very good proportion of the data structures that you will use in Python. We encourage you to take a look at the Python documentation and experiment further with each and every data type we've seen in this chapter. It's worth it—believe us. Everything you'll write will be about handling data, so make sure your knowledge about it is rock solid.

Before we leap intoChapter 3,Conditionals and Iteration, we'd like to share some final considerations about different aspects that, to our minds, are important and not to be neglected.

Small value caching

While discussing objects at thebeginning of this chapter, we saw that when we assigned a name to an object, Python creates the object, sets its value, and then points the name to it. We can assign different names to the same value, and we expect different objects to be created, like this:

>>> a =1000000>>> b =1000000>>>id(a) ==id(b)False

In the preceding example,a andb are assigned to twoint objects, which have the same value, but they are not the same object—as you can see, theirid is not the same. So, let's do it again:

>>> a =5>>> b =5>>>id(a) ==id(b)True

Uh-oh! Is Python broken? Why are the two objects the same now? We didn't doa = b = 5; we set them up separately.

Well, the answer isperformance. Python caches short strings and small numbers to avoid having many copies of them clogging up the system memory. In the case of strings, caching or, more appropriately, interning them, also provides a significant performance improvement for comparison operations. Everything is handled properly under the hood, so you don't need to worry, but make sure that you remember this behavior should your code ever need to fiddle with IDs.

How to choose data structures

As we've seen, Python provides you with several built-in data types and, sometimes, if you're not that experienced, choosing the one that serves you best can be tricky, especially when it comes to collections. For example, say you have many dictionaries to store, each of which represents a customer. Within each customer dictionary, there's an'id': 'code' unique identification code. In what kind of collection would you place them? Well, unless we know more about these customers, it's very hard to answer. What kind of access will we need? What sort of operations will we have to perform on each of them, and how many times? Will the collection change over time? Will we need to modify the customer dictionaries in any way? What is going to be the most frequent operation we will have to perform on the collection?

If you can answer the preceding questions, then you will know what to choose. If the collection never shrinks or grows (in other words, it won't need to add/delete any customer object after creation) or shuffles, then tuples are a possible choice. Otherwise, lists are a good candidate. Every customer dictionary has a unique identifier though, so even a dictionary could work. Let us draft these options for you:

# example customer objectscustomer1 = {'id':'abc123','full_name':'Master Yoda'}customer2 = {'id':'def456','full_name':'Obi-Wan Kenobi'}customer3 = {'id':'ghi789','full_name':'Anakin Skywalker'}# collect them in a tuplecustomers = (customer1, customer2, customer3)# or collect them in a listcustomers = [customer1, customer2, customer3]# or maybe within a dictionary, they have a unique id after allcustomers = {'abc123': customer1,'def456': customer2,'ghi789': customer3,}

Some customers we have there, right? We probably wouldn't go with the tuple option, unless we wanted to highlight that the collection is not going to change. We would say that, usually, a list is better, as it allows for more flexibility.

Another factor to keep inmind is that tuples and lists are ordered collections. If you use a dictionary (prior to Python 3.6) or a set, you would lose the ordering, so you need to know if ordering is important in your application.

What about performance? For example, in a list, operations such as insertion and membership testing can takeO(n) time, while they areO(1) for a dictionary. It's not always possible to use dictionaries though, if we don't have the guarantee that we can uniquely identify each item of the collection by means of one of its properties, and that the property in question is hashable (so it can be a key indict).

Another way of understanding whether you have chosen the right data structure is by looking at the code you have to write in order to manipulate it. If everything comes easily and flows naturally, then you probably have chosen correctly, but if you find yourself thinking your code is getting unnecessarily complicated, then you probably should try to decide whether you need to reconsider your choices. It's quite hard to give advice without a practical case though, so when you choose a data structure for your data, try to keep ease of use and performance in mind, and give precedence to what matters most in the context you are in.

About indexing and slicing

At the beginning of this chapter, we saw slicing applied to strings. Slicing, in general, applies to a sequence: tuples, lists, strings, and so on. With lists, slicing can also be used for assignment. We have almost never seen this used in professional code, but still, you know you can. Could you slice dictionaries or sets? We hear you scream,Of course not! Excellent—we see that we're on the same page here, so let's talk about indexing.

There is one characteristic regarding Python indexing that we haven't mentioned before. We'll show you by way of an example. How do you address the last element of a collection? Let's see:

>>> a =list(range(10))# `a` has 10 elements. Last one is 9.>>> a[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]>>>len(a)# its length is 10 elements10>>> a[len(a) -1]# position of last one is len(a) - 19>>> a[-1]# but we don't need len(a)! Python rocks!9>>> a[-2]# equivalent to len(a) - 28>>> a[-3]# equivalent to len(a) - 37

If lista has 10 elements, then due to the 0-index positioning system of Python, the first one is at position 0 and the last one is at position 9. In the preceding example, the elements are conveniently placed in a position equal to their value:0 is at position 0,1 at position 1, and so on.

So, in order to fetch the last element, we need toknow the length of the whole list (or tuple, or string, and so on) and then subtract 1. Hence:len(a) - 1. This is so common an operation that Python provides you with a way to retrieve elements usingnegative indexing. This proves very useful when performing data manipulation.Figure 2.2 displays a neat diagram about how indexing works on the string"HelloThere" (which is Obi-Wan Kenobi sarcastically greeting General Grievous inStar Wars: Episode III – Revenge of the Sith):

Figure 2.2: Python indexing

Trying to address indexes greater than 9 or smaller than -10 will raise anIndexError, as expected.

About names

You may have noticed that, in order to keep theexamples as short as possible, we have named many objects using simple letters, likea,b,c,d, and so on. This is perfectly fine when debugging on the console or showing thata + b == 7, but it's bad practice when it comes to professional coding (or any type of coding, for that matter). We hope you will indulge us where we have done it; the reason is to present the code in a more compact way.

In a real environment though, when you choose names for your data, you should choose them carefully—they should reflect what the data is about. So, if you have a collection ofCustomer objects,customers is a perfectly good name for it. Wouldcustomers_list,customers_tuple, orcustomers_collection work as well? Think about it for a second. Is it good to tie the name of the collection to the datatype? We don't think so, at least in most cases. So, if you have an excellent reason to do so, go ahead; otherwise, don't. The reasoning behind this is that oncecustomers_tuple starts being used in different places of your code, and you realize you actually want to use a list instead of a tuple, you're up for some fun refactoring (also known aswasted time). Names for data should be nouns, and names for functions should be verbs. Names should be as expressive as possible. Python is actually a very goodexample when it comes to names. Most of the time you can just guess what a function is called if you know what it does. Crazy, huh?

Chapter 2 from the bookClean Code byRobert C. Martinis entirely dedicated to names. It's an amazing book that helped us improve our coding style in many different ways; it is a must-read if you want to take your coding to the next level.

In this chapter, we've explored the built-in data types of Python. We've seen how many there are and how much can be achieved just by using them in different combinations.

We've seen number types, sequences, sets, mappings, dates, times, and collections (and a special guest appearance by enumerations). We have also seen that everything is an object, learned the difference between mutable and immutable, and we've also learned about slicing and indexing.

We've presented the cases with simple examples, but there's much more that you can learn about this subject, so stick your nose into the official documentation and go exploring!

Most of all, we encourage you to try out all of the exercises by yourself—get your fingers using that code, build some muscle memory, and experiment, experiment, experiment. Learn what happens when you divide by zero, when you combine different number types into a single expression, and when you massage strings. Play with all data types. Exercise them, break them, discover all their methods, enjoy them, and learn them very, very well. If your foundation is not rock solid, how good can your code be? Data is the foundation for everything; data shapes what dances around it.

The more you progress with the book, the more likely it is that you will find some discrepancies or maybe a small typo here and there in our code (or yours). You will get an error message, something will break. That's wonderful! When you code, things break and you have to debug them, all the time, so consider errors as useful exercises to learn something new about the language you're using, and not as failures or problems. Errors will keep coming up, that's for sure, so you may as well start making your peace with them now.

The next chapter is about conditionals and iteration. We'll see how to actually put collections to use, and make decisions based on the data that we're presented with. We'll start to go a little faster now that your knowledge is building up, so make sure you're comfortable with the contents of this chapter before you move to the next one. Once more, have fun, explore, and break things—it's a very good way to learn.