Variables in C

Let’s say you had the following code that defines the variablex:

intx=2337;

This one line of code has several, distinct steps when executed:

1. Allocate enough memory for an integer
2. Assign the value2337 to that memory location
3. Indicate thatx points to that value

Shown in a simplified view of memory, it might look like this:

Here, you can see that the variablex has a fake memory location of0x7f1 and the value2337. If, later in the program, you want to change the value ofx, you can do the following:

x=2338;

The above code assigns a new value (2338) to the variablex, therebyoverwriting the previous value. This means that the variablex ismutable. The updated memory layout shows the new value:

Notice that the location ofx didn’t change, just the value itself. This is a significant point. It means thatxis the memory location, not just a name for it.

Another way to think of this concept is in terms of ownership. In one sense,x owns the memory location.x is, at first, an empty box that can fit exactly one integer in which integer values can be stored.

When you assign a value tox, you’re placing a value in the box thatx owns. If you wanted to introduce a new variable (y), you could add this line of code:

inty=x;

This code creates anew box calledy and copies the value fromx into the box. Now the memory layout will look like this:

Notice the new location0x7f5 ofy. Even though the value ofx was copied toy, the variabley owns some new address in memory. Therefore, you could overwrite the value ofy without affectingx:

y=2339;

Now the memory layout will look like this:

Again, you have modified the value aty, butnot its location. In addition, you have not affected the originalx variable at all. This is in stark contrast with how Python names work.

Names in Python

Python does not have variables. It has names. Yes, this is a pedantic point, and you can certainly use the term variables as much as you like. It is important to know that there is a difference between variables and names.

Let’s take the equivalent code from the above C example and write it in Python:

>>>x=2337

Much like in C, the above code is broken down into several distinct steps during execution:

1. Create aPyObject
2. Set the typecode to integer for thePyObject
3. Set the value to2337 for thePyObject
4. Create a name calledx
5. Pointx to the newPyObject
6. Increase the refcount of thePyObject by 1

In memory, it might looks something like this:

You can see that the memory layout is vastly different than the C layout from before. Instead ofx owning the block of memory where the value2337 resides, the newly created Python object owns the memory where2337 lives. The Python namex doesn’t directly ownany memory address in the way the C variablex owned a static slot in memory.

If you were to try to assign a new value tox, you could try the following:

>>>x=2338

What’s happening here is different than the C equivalent, but not too different from the original bind in Python.

This code:

• Creates a newPyObject
• Sets the typecode to integer for thePyObject
• Sets the value to2338 for thePyObject
• Pointsx to the newPyObject
• Increases the refcount of the newPyObject by 1
• Decreases the refcount of the oldPyObject by 1

Now in memory, it would look something like this:

This diagram helps illustrate thatx points to a reference to an object and doesn’t own the memory space as before. It also shows that thex = 2338 command is not an assignment, but rather binding the namex to a reference.

In addition, the previous object (which held the2337 value) is now sitting in memory with a ref count of 0 and will get cleaned up by thegarbage collector.

You could introduce a new name,y, to the mix as in the C example:

>>>y=x

In memory, you would have a new name, but not necessarily a new object:

Now you can see that a new Python object hasnot been created, just a new name that points to the same object. Also, the object’s refcount has increased by one. You could check for object identity equality to confirm that they are the same:

>>>yisxTrue

The above code indicates thatx andy are the same object. Make no mistake though:y is still immutable.

For example, you could perform addition ony:

>>>y+=1>>>yisxFalse

After the addition call, you are returned with a new Python object. Now, the memory looks like this:

A new object has been created, andy now points to the new object. Interestingly, this is the same end-state if you had boundy to2339 directly:

>>>y=2339

The above statement results in the same end-memory state as the addition. To recap, in Python, you don’t assign variables. Instead, you bind names to references.

A Note on Intern Objects in Python

Now that you understand how Python objects get created and names get bound to those objects, its time to throw a wrench in the machinery. That wrench goes by the name of interned objects.

Suppose you have the following Python code:

>>>x=1000>>>y=x>>>xisyTrue

As above,x andy are both names that point to the same Python object. But the Python object that holds the value1000 is not always guaranteed to have the same memory address. For example, if you were to assign a literal1000 toy as well, you would end up with a different memory address:

>>>x=1000>>>y=1000>>>xisyFalse

This time, the linex is y returnsFalse. If this is confusing, then don’t worry. Here are the steps that occur when this code is executed:

1. Create a Python object (1000)
2. Assign the namex to that object
3. Create a new Python object (1000)
4. Assign the namey to that object

Isn’t this wasteful? Well, yes it is, but that’s the price you pay for all of the great benefits of Python. You never have to worry about cleaning up these intermediate objects or even need to know that they exist! The joy is that these operations are relatively fast, and you never had to know any of those details until now.

The core Python developers, in their wisdom, also noticed this waste and decided to make a few optimizations. These optimizations result in behavior that can be surprising to newcomers:

>>>x=20>>>y=20>>>xisyTrue

In this example, you see nearly the same code as before, except this time the result isTrue. This is the result of interned objects. Python pre-creates a certain subset of objects in memory and keeps them in the globalnamespace for everyday use.

Which objects depend on the implementation of Python. CPython 3.7 interns the following:

1. Integer numbers between-5 and256
2. Strings that contain ASCII letters (A toZ anda toz), digits, and underscores only

The reasoning behind this is that these objects are likely to be used in many programs. For example, most variable names in Python are covered by the second bullet point. By interning these objects, Python prevents memory allocation calls for consistently used objects.

Strings that are less than 20 characters and contain ASCII letters, digits, or underscores will be interned. This can make these strings more effective to use in comparisons:

>>>s1="realpython">>>id(s1)140696485006960>>>s2="realpython">>>id(s2)140696485006960>>>s1iss2True

Here you can see thats1 ands2 both point to the same address in memory. If you were to introduce a character that isn’t an ASCII letter, digit, or underscore, then you would get a different result:

>>>s1="Real Python!">>>s2="Real Python!">>>s1iss2False

Because this example has an exclamation mark (!) in it, these strings are not interned and are different objects in memory.

Interned objects are often a source of confusion. Just remember, if you’re ever in doubt, that you can always useid() andis to determine object equality.

Using Mutable Types as Pointers

You’ve already learned about mutable types. Because these objects are mutable, you can treat them as if they were pointers to simulate pointer behavior. Suppose you wanted to replicate the following c code:

voidadd_one(int*x){*x+=1;}

This code takes a pointer to an integer (*x) and then increments the value by one. Here is a main function to exercise the code:

#include<stdio.h>intmain(void){inty=2337;printf("y = %d\n",y);add_one(&y);printf("y = %d\n",y);return0;}

In the above code, you assign2337 toy,print out the current value, increment the value by one, and then print out the modified value. The output of executing this code would be the following:

y = 2337y = 2338

One way to replicate this type of behavior in Python is by using a mutable type. Consider using a list and modifying the first element:

>>>defadd_one(x):...x[0]+=1...>>>y=[2337]>>>add_one(y)>>>y[0]2338

Here,add_one(x) accesses the first element and increments its value by one. Using alist means that the end result appears to have modified the value. So pointers in Python do exist? Well, no. This is only possible becauselist is a mutable type. If you tried to use atuple, you would get an error:

>>>z=(2337,)>>>add_one(z)Traceback (most recent call last):  File"<stdin>", line1, in<module>  File"<stdin>", line2, inadd_oneTypeError:'tuple' object does not support item assignment

The above code demonstrates thattuple is immutable. Therefore, it does not support item assignment.list is not the only mutable type. Another common approach to mimicking pointers in Python is to use adict.

Let’s say you had an application where you wanted to keep track of every time an interesting event happened. One way to achieve this would be to create adict and use one of the items as a counter:

>>>counters={"func_calls":0}>>>defbar():...counters["func_calls"]+=1...>>>deffoo():...counters["func_calls"]+=1...bar()...>>>foo()>>>counters["func_calls"]2

In this example, thecountersdictionary is used to keep track of the number of function calls. After you callfoo(), the counter has increased to2 as expected. All becausedict is mutable.

Keep in mind, this is onlysimulates pointer behavior and does not directly map to true pointers in C or C++. That is to say, these operations are more expensive than they would be in C or C++.

Using Python Objects

Thedict option is a great way to emulate pointers in Python, but sometimes it gets tedious to remember the key name you used. This is especially true if you’re using the dictionary in various parts of your application. This is where a customPython class can really help.

To build on the last example, assume that you want to track metrics in your application. Creating a class is a great way to abstract the pesky details:

classMetrics(object):def__init__(self):self._metrics={"func_calls":0,"cat_pictures_served":0,}

This code defines aMetrics class. This class still uses adict for holding the actual data, which is in the_metrics member variable. This will give you the mutability you need. Now you just need to be able to access these values. One nice way to do this is with properties:

classMetrics(object):# ...@propertydeffunc_calls(self):returnself._metrics["func_calls"]@propertydefcat_pictures_served(self):returnself._metrics["cat_pictures_served"]

This code makes use of@property. If you’re not familiar with decorators, you can check out thisPrimer on Python Decorators. The@property decorator here allows you to accessfunc_calls andcat_pictures_served as if they were attributes:

>>>metrics=Metrics()>>>metrics.func_calls0>>>metrics.cat_pictures_served0

The fact that you can access these names as attributes means that you abstracted the fact that these values are in adict. You also make it more explicit what the names of the attributes are. Of course, you need to be able to increment these values:

classMetrics(object):# ...definc_func_calls(self):self._metrics["func_calls"]+=1definc_cat_pics(self):self._metrics["cat_pictures_served"]+=1

You have introduced two new methods:

1. inc_func_calls()
2. inc_cat_pics()

These methods modify the values in the metricsdict. You now have a class that you modify as if you’re modifying a pointer:

>>>metrics=Metrics()>>>metrics.inc_func_calls()>>>metrics.inc_func_calls()>>>metrics.func_calls2

Here, you can accessfunc_calls and callinc_func_calls() in various places in your applications and simulate pointers in Python. This is useful when you have something like metrics that need to be used and updated frequently in various parts of your applications.

Here’s the full source for theMetrics class:

classMetrics(object):def__init__(self):self._metrics={"func_calls":0,"cat_pictures_served":0,}@propertydeffunc_calls(self):returnself._metrics["func_calls"]@propertydefcat_pictures_served(self):returnself._metrics["cat_pictures_served"]definc_func_calls(self):self._metrics["func_calls"]+=1definc_cat_pics(self):self._metrics["cat_pictures_served"]+=1