The Walrus in the Room: Assignment Expressions

The biggest change in Python 3.8 is the introduction ofassignment expressions. They are written using a new notation (:=). This operator is often called thewalrus operator as it resembles the eyes and tusks of a walrus on its side.

Assignment expressions allow you to assign and return a value in the same expression. For example, if you want to assign to avariable andprint its value, then you typically do something like this:

>>>walrus=False>>>print(walrus)False

In Python 3.8, you’re allowed to combine these two statements into one, using the walrus operator:

>>>print(walrus:=True)True

The assignment expression allows you to assignTrue towalrus, and immediately print the value. But keep in mind that the walrus operator doesnot do anything that isn’t possible without it. It only makes certain constructs more convenient, and can sometimes communicate the intent of your code more clearly.

One pattern that shows some of the strengths of the walrus operator iswhile loops where you need to initialize and update a variable. For example, the following code asks the user for input until they typequit:

inputs=list()current=input("Write something: ")whilecurrent!="quit":inputs.append(current)current=input("Write something: ")

This code is less than ideal. You’re repeating theinput() statement, and somehow you need to addcurrent to the listbefore asking the user for it. A better solution is to set up an infinitewhile loop, and usebreak to stop the loop:

inputs=list()whileTrue:current=input("Write something: ")ifcurrent=="quit":breakinputs.append(current)

This code is equivalent to the one above, but avoids the repetition and somehow keeps the lines in a more logical order. If you use an assignment expression, you can simplify this loop further:

inputs=list()while(current:=input("Write something: "))!="quit":inputs.append(current)

This moves the test back to thewhile line, where it should be. However, there are now several things happening at that line, so it takes a bit more effort to read it properly. Use your best judgement about when the walrus operator helps make your code more readable.

PEP 572 describes all the details of assignment expressions, including some of the rationale for introducing them into the language, as well asseveral examples of how the walrus operator can be used.

Positional-Only Arguments

The built-in functionfloat() can be used for convertingtext strings and numbers tofloat objects. Consider the following example:

>>>float("3.8")3.8>>>help(float)class float(object) |  float(x=0, /) | |  Convert a string or number to a floating point number, if possible.[...]

Look closely at the signature offloat(). Notice the slash (/) after the parameter. What does it mean?

It turns out that while the one parameter offloat() is calledx, you’re not allowed to use its name:

>>>float(x="3.8")Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:float() takes no keyword arguments

When usingfloat() you’re only allowed to specify arguments by position, not by keyword. Before Python 3.8, suchpositional-only arguments were only possible for built-in functions. There was no easy way to specify that arguments should be positional-only in your own functions:

>>>defincr(x):...returnx+1...>>>incr(3.8)4.8>>>incr(x=3.8)4.8

It’s possible tosimulate positional-only argumentsusing*args, but this is less flexible, less readable, and forces you to implement your own argument parsing. In Python 3.8, you can use/ to denote that all arguments before it must be specified by position. You can rewriteincr() to only accept positional arguments:

>>>defincr(x,/):...returnx+1...>>>incr(3.8)4.8>>>incr(x=3.8)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:incr() got some positional-only arguments passed as           keyword arguments: 'x'

By adding/ afterx, you specify thatx is a positional-only argument. You can combine regular arguments with positional-only ones by placing the regular arguments after the slash:

>>>defgreet(name,/,greeting="Hello"):...returnf"{greeting},{name}"...>>>greet("Łukasz")'Hello, Łukasz'>>>greet("Łukasz",greeting="Awesome job")'Awesome job, Łukasz'>>>greet(name="Łukasz",greeting="Awesome job")Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:greet() got some positional-only arguments passed as           keyword arguments: 'name'

Ingreet(), the slash is placed betweenname andgreeting. This means thatname is a positional-only argument, whilegreeting is a regular argument that can be passed either by position or by keyword.

At first glance, positional-only arguments can seem a bit limiting and contrary to Python’s mantra about the importance of readability. You will probably find that there are not a lot of occasions where positional-only arguments improve your code.

However, in the right circumstances, positional-only arguments can give you some flexibility when you’re designing functions. First, positional-only arguments make sense when you have arguments that have a natural order but are hard to give good, descriptive names to.

Another possible benefit of using positional-only arguments is that you can more easily refactor your functions. In particular, you can change the name of your parameters without worrying that other code depends on those names.

Positional-only arguments nicely complementkeyword-only arguments. In any version of Python 3, you can specify keyword-only arguments using the star (*). Any argumentafter* must be specified using a keyword:

>>>defto_fahrenheit(*,celsius):...return32+celsius*9/5...>>>to_fahrenheit(40)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:to_fahrenheit() takes 0 positional arguments but 1 was given>>>to_fahrenheit(celsius=40)104.0

celsius is a keyword-only argument, so Python raises an error if you try to specify it based on position, without the keyword.

You can combine positional-only, regular, and keyword-only arguments, by specifying them in this order separated by/ and*. In the following example,text is a positional-only argument,border is a regular argument with a default value, andwidth is a keyword-only argument with a default value:

>>>defheadline(text,/,border="♦",*,width=50):...returnf"{text} ".center(width,border)...

Sincetext is positional-only, you can’t use the keywordtext:

>>>headline("Positional-only Arguments")'♦♦♦♦♦♦♦♦♦♦♦ Positional-only Arguments ♦♦♦♦♦♦♦♦♦♦♦♦'>>>headline(text="This doesn't work!")Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:headline() got some positional-only arguments passed as           keyword arguments: 'text'

border, on the other hand, can be specified both with and without the keyword:

>>>headline("Python 3.8","=")'=================== Python 3.8 ==================='>>>headline("Real Python",border=":")':::::::::::::::::: Real Python :::::::::::::::::::'

Finally,width must be specified using the keyword:

>>>headline("Python","🐍",width=38)'🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍 Python 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍'>>>headline("Python","🐍",38)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:headline() takes from 1 to 2 positional arguments           but 3 were given

You can read more about positional-only arguments inPEP 570.

More Precise Types

Python’s typing system is quite mature at this point. However, in Python 3.8, some new features have been added totyping to allow more precise typing:

• Literal types
• Typed dictionaries
• Final objects
• Protocols

Python supports optionaltype hints, typically as annotations on your code:

defdouble(number:float)->float:return2*number

In this example, you say thatnumber should be afloat and thedouble() function should return afloat, as well. However, Python treats these annotations ashints. They are not enforced at runtime:

>>>double(3.14)6.28>>>double("I'm not a float")"I'm not a floatI'm not a float"

double() happily accepts"I'm not a float" as an argument, even though that’s not afloat. There arelibraries that can use types at runtime, but that is not the main use case for Python’s type system.

Instead, type hints allowstatic type checkers to do type checking of your Python code, without actually running your scripts. This is reminiscent of compilers catching type errors in other languages likeJava,Rust, andCrystal. Additionally, type hints act asdocumentation of your code, making it easier to read, as well asimproving auto-complete in your IDE.

$python-mpipinstallmypy

You can find more information about type hints in Python in theoriginal PEP 484, as well as inPython Type Checking (Guide).

There are four new PEPs about type checking that have been accepted and included in Python 3.8. You’ll see short examples from each of these.

PEP 586 introduce theLiteral type.Literal is a bit special in that it represents one or several specific values. One use case ofLiteral is to be able to precisely add types, when string arguments are used to describe specific behavior. Consider the following example:

# draw_line.pydefdraw_line(direction:str)->None:ifdirection=="horizontal":...# Draw horizontal lineelifdirection=="vertical":...# Draw vertical lineelse:raiseValueError(f"invalid direction{direction!r}")draw_line("up")

The program will pass the static type checker, even though"up" is an invalid direction. The type checker only checks that"up" is a string. In this case, it would be more precise to say thatdirection must be either the literal string"horizontal" or the literal string"vertical". UsingLiteral, you can do exactly that:

# draw_line.pyfromtypingimportLiteraldefdraw_line(direction:Literal["horizontal","vertical"])->None:ifdirection=="horizontal":...# Draw horizontal lineelifdirection=="vertical":...# Draw vertical lineelse:raiseValueError(f"invalid direction{direction!r}")draw_line("up")

By exposing the allowed values ofdirection to the type checker, you can now be warned about the error:

$mypydraw_line.pydraw_line.py:15: error:    Argument 1 to "draw_line" has incompatible type "Literal['up']";    expected "Union[Literal['horizontal'], Literal['vertical']]"Found 1 error in 1 file (checked 1 source file)

The basic syntax isLiteral[<literal>]. For instance,Literal[38] represents the literal value 38. You can express one of several literal values usingUnion:

Union[Literal["horizontal"],Literal["vertical"]]

Since this is a fairly common use case, you can (and probably should) use the simpler notationLiteral["horizontal", "vertical"] instead. You already used the latter when adding types todraw_line(). If you look carefully at the output from Mypy above, you can see that it translated the simpler notation to theUnion notation internally.

There are cases where the type of the return value of a function depends on the input arguments. One example isopen() which may return a text string or a byte array depending on the value ofmode. This can be handled throughoverloading.

The following example shows the skeleton of a calculator that can return the answer either as regular numbers (38), or asroman numerals (XXXVIII):

# calculator.pyfromtypingimportUnionARABIC_TO_ROMAN=[(1000,"M"),(900,"CM"),(500,"D"),(400,"CD"),(100,"C"),(90,"XC"),(50,"L"),(40,"XL"),(10,"X"),(9,"IX"),(5,"V"),(4,"IV"),(1,"I")]def_convert_to_roman_numeral(number:int)->str:"""Convert number to a roman numeral string"""result=list()forarabic,romaninARABIC_TO_ROMAN:count,number=divmod(number,arabic)result.append(roman*count)return"".join(result)defadd(num_1:int,num_2:int,to_roman:bool=True)->Union[str,int]:"""Add two numbers"""result=num_1+num_2ifto_roman:return_convert_to_roman_numeral(result)else:returnresult

The code has the correct type hints: the result ofadd() will be eitherstr orint. However, often this code will be called with a literalTrue orFalse as the value ofto_roman in which case you would like the type checker to infer exactly whetherstr orint is returned. This can be done usingLiteral together with@overload:

# calculator.pyfromtypingimportLiteral,overload,UnionARABIC_TO_ROMAN=[(1000,"M"),(900,"CM"),(500,"D"),(400,"CD"),(100,"C"),(90,"XC"),(50,"L"),(40,"XL"),(10,"X"),(9,"IX"),(5,"V"),(4,"IV"),(1,"I")]def_convert_to_roman_numeral(number:int)->str:"""Convert number to a roman numeral string"""result=list()forarabic,romaninARABIC_TO_ROMAN:count,number=divmod(number,arabic)result.append(roman*count)return"".join(result)@overloaddefadd(num_1:int,num_2:int,to_roman:Literal[True])->str:...@overloaddefadd(num_1:int,num_2:int,to_roman:Literal[False])->int:...defadd(num_1:int,num_2:int,to_roman:bool=True)->Union[str,int]:"""Add two numbers"""result=num_1+num_2ifto_roman:return_convert_to_roman_numeral(result)else:returnresult

The added@overload signatures will help your type checker inferstr orint depending on the literal values ofto_roman. Note that the ellipses (...) are a literal part of the code. They stand in for the function body in the overloaded signatures.

As a complement toLiteral,PEP 591 introducesFinal. This qualifier specifies that a variable or attribute should not be reassigned, redefined, or overridden. The following is a typing error:

fromtypingimportFinalID:Final=1...ID+=1

Mypy will highlight the lineID += 1, and note that youCannot assign to final name "ID". This gives you a way to ensure that constants in your code never change their value.

Additionally, there is also a@final decorator that can be applied to classes and methods. Classesdecorated with@final can’t be subclassed, while@final methods can’t be overridden by subclasses:

fromtypingimportfinal@finalclassBase:...classSub(Base):...

Mypy will flag this example with the error messageCannot inherit from final class "Base". To learn more aboutFinal and@final, seePEP 591.

The third PEP allowing for more specific type hints isPEP 589, which introducesTypedDict. This can be used to specify types for keys and values in a dictionary using a notation that is similar to the typedNamedTuple.

Traditionally, dictionaries have been annotated usingDict. The issue is that this only allowed one type for the keys and one type for the values, often leading to annotations likeDict[str, Any]. As an example, consider a dictionary that registers information about Python versions:

py38={"version":"3.8","release_year":2019}

The value corresponding toversion is a string, whilerelease_year is an integer. This can’t be precisely represented usingDict. With the newTypedDict, you can do the following:

fromtypingimportTypedDictclassPythonVersion(TypedDict):version:strrelease_year:intpy38=PythonVersion(version="3.8",release_year=2019)

The type checker will then be able to infer thatpy38["version"] has typestr, whilepy38["release_year"] is anint. At runtime, aTypedDict is a regulardict, and type hints are ignored as usual. You can also useTypedDict purely as an annotation:

py38:PythonVersion={"version":"3.8","release_year":2019}

Mypy will let you know if any of your values has the wrong type, or if you use a key that has not been declared. SeePEP 589 for more examples.

Mypy has supportedProtocols for a while already. However, theofficial acceptance only happened in May 2019.

Protocols are a way of formalizing Python’s support for duck typing:

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck. (Source)

Duck typing allows you to, for example, read.name on any object that has a.name attribute, without really caring about the type of the object. It may seem counter-intuitive for the typing system to support this. Throughstructural subtyping, it’s still possible to make sense of duck typing.

You can for instance define a protocol calledNamed that can identify all objects with a.name attribute:

fromtypingimportProtocolclassNamed(Protocol):name:strdefgreet(obj:Named)->None:print(f"Hi{obj.name}")

Here,greet() takes any object, as long as it defines a.name attribute. SeePEP 544 andthe Mypy documentation for more information about protocols.

Simpler Debugging With f-Strings

f-strings were introduced in Python 3.6, and have become very popular. They might be the most common reason for Python libraries only being supported on version 3.6 and later. An f-string is a formatted string literal. You can recognize it by the leadingf:

>>>style="formatted">>>f"This is a{style} string"'This is a formatted string'

When you use f-strings, you can enclose variables and even expressions inside curly braces. They will then be evaluated at runtime and included in the string. You can have several expressions in one f-string:

>>>importmath>>>r=3.6>>>f"A circle with radius{r} has area{math.pi*r*r:.2f}"'A circle with radius 3.6 has area 40.72'

In the last expression,{math.pi * r * r:.2f}, you also use a format specifier. Format specifiers are separated from the expressions with a colon.

.2f means that the area is formatted as a floating point number with 2 decimals. The format specifiers are the same as for.format(). See theofficial documentation for a full list of allowed format specifiers.

In Python 3.8, you can use assignment expressions inside f-strings. Just make sure to surround the assignment expression with parentheses:

>>>importmath>>>r=3.8>>>f"Diameter{(diam:=2*r)} gives circumference{math.pi*diam:.2f}"'Diameter 7.6 gives circumference 23.88'

However, the real f-news in Python 3.8 is the new debugging specifier. You can now add= at the end of an expression, and it will print both the expression and its value:

>>>python=3.8>>>f"{python=}"'python=3.8'

This is a short-hand, that typically will be most useful when working interactively or adding print statements to debug your script. In earlier versions of Python, you needed to spell out the variable or expression twice to get the same information:

>>>python=3.7>>>f"python={python}"'python=3.7'

You can add spaces around=, and use format specifiers as usual:

>>>name="Eric">>>f"{name= }""name = 'Eric'">>>f"{name= :>10}"'name =       Eric'

The>10 format specifier says thatname should be right-aligned within a 10 character string.= works for more complex expressions as well:

>>>f"{name.upper()[::-1]= }""name.upper()[::-1] = 'CIRE'"

For more information about f-strings, seePython’s F-String for String Interpolation and Formatting.

The Python Steering Council

Technically,Python’sgovernance is not a language feature. However, Python 3.8 is the first version of Python not developed under thebenevolent dictatorship ofGuido van Rossum. The Python language is now governed by asteering council consisting of five core developers:

• Barry Warsaw
• Brett Cannon
• Carol Willing
• Guido van Rossum
• Nick Coghlan

The road to the new governance model for Python was an interesting study in self-organization. Guido van Rossum created Python in the early 1990s, and has been affectionally dubbed Python’sBenevolent Dictator for Life (BDFL). Through the years, more and more decisions about the Python language were made throughPython Enhancement Proposals (PEPs). Still, Guido officially had the last word on any new language feature.

After a long and drawn out discussion aboutassignment expressions, Guidoannounced in July 2018 that he was retiring from his role as BDFL (for real this time). He purposefully did not name a successor. Instead, he asked the team of core developers to figure out how Python should be governed going forward.

Luckily, the PEP process was already well established, so it was natural to use PEPs to discuss and decide on a new governance model. Through the fall of 2018,several models were proposed, includingelecting a new BDFL (renamed the Gracious Umpire Influencing Decisions Officer: the GUIDO), or moving to acommunity model based on consensus and voting, without centralized leadership. In December 2018, thesteering council model was chosen after a vote among the core developers.

The steering council consists of five members of the Python community, as listed above. There will be an election for a new steering council after every major release of Python. In other words, there will be an election following the release of Python 3.8.

Although it’s an open election, it’s expected that most, if not all, of the inaugural steering council will be reelected. The steering council hasbroad powers to make decisions about the Python language, but should strive to exercise those powers as little as possible.

You can read all about the new governance model inPEP 13, while the process of deciding on the new model is described inPEP 8000. For more information, see thePyCon 2019 Keynote, and listen to Brett Cannon onTalk Python To Me and onThe Changelog podcast. You can follow updates from the steering council onGitHub.

Other Pretty Cool Features

So far, you’ve seen the headline news regarding what’s new in Python 3.8. However, there are many other changes that are also pretty cool. In this section, you’ll get a quick look at some of them.

>>>fromimportlibimportmetadata>>>metadata.version("pip")'19.2.3'>>>pip_metadata=metadata.metadata("pip")>>>list(pip_metadata)['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'License', 'Keywords', 'Platform', 'Classifier',  'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier',  'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier',  'Classifier', 'Classifier', 'Requires-Python']>>>pip_metadata["Home-page"]'https://pip.pypa.io/'>>>pip_metadata["Requires-Python"]'>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*'>>>len(metadata.files("pip"))668

>>>[pforpinmetadata.files("realpython-reader")ifp.suffix==".py"][PackagePath('reader/__init__.py'), PackagePath('reader/__main__.py'), PackagePath('reader/feed.py'), PackagePath('reader/viewer.py')]>>>init_path=_[0]# Underscore access last returned value in the REPL>>>print(init_path.read_text())"""Real Python feed readerImport the `feed` module to work with the Real Python feed:    >>> from reader import feed    >>> feed.get_titles()    ['Logging in Python', 'The Best Python Books', ...]See https://github.com/realpython/reader/ for more information"""# Version of realpython-reader package__version__ = "1.0.0"...

>>>metadata.requires("realpython-reader")['feedparser', 'html2text', 'importlib-resources', 'typing']

$python-mpipinstallimportlib-metadata

try:fromimportlibimportmetadataexceptImportError:importimportlib_metadataasmetadata...

>>>importmath>>>math.prod((2,8,7,7))784>>>2*8*7*7784

>>>importmath>>>math.isqrt(9)3>>>math.sqrt(9)3.0>>>math.isqrt(15)3>>>math.sqrt(15)3.872983346207417

>>>importmath>>>point_1=(16,25,20)>>>point_2=(8,15,14)>>>math.dist(point_1,point_2)14.142135623730951>>>math.hypot(*point_1)35.79106033634656>>>math.hypot(*point_2)22.02271554554524

>>>importstatistics>>>data=[9,3,2,1,1,2,7,9]>>>statistics.fmean(data)4.25>>>statistics.geometric_mean(data)3.013668912157617>>>statistics.multimode(data)[9, 2, 1]>>>statistics.quantiles(data,n=4)[1.25, 2.5, 8.5]

>>>importrandom>>>importstatistics>>>fromtimeitimporttimeit>>># Create 10,000 random numbers>>>data=[random.random()for_inrange(10_000)]>>># Measure the time it takes to run mean() and fmean()>>>t_mean=[timeit("statistics.mean(data)",number=100,globals=globals())...for_inrange(30)]>>>t_fmean=[timeit("statistics.fmean(data)",number=100,globals=globals())...for_inrange(30)]>>># Create NormalDist objects based on the sampled timings>>>n_mean=statistics.NormalDist.from_samples(t_mean)>>>n_fmean=statistics.NormalDist.from_samples(t_fmean)>>># Look at sample mean and standard deviation>>>n_mean.mean,n_mean.stdev(0.825690647733245, 0.07788573997674526)>>>n_fmean.mean,n_fmean.stdev(0.010488564966666065, 0.0008572332785645231)>>># Calculate the lower 1 percentile of mean>>>n_mean.quantiles(n=100)[0]0.6445013221202459

>>># Python 3.7>>>version="3.7">>>versionis"3.7"False>>># Python 3.8>>>version="3.8">>>versionis"3.8"<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?False>>>version=="3.8"True

>>>[...(1,3)...(2,4)...]<stdin>:2: SyntaxWarning: 'tuple' object is not callable; perhaps           you missed a comma?Traceback (most recent call last):  File"<stdin>", line2, in<module>TypeError:'tuple' object is not callable

>>>importcollections>>>fromtimeitimporttimeit>>>Person=collections.namedtuple("Person","name twitter")>>>raymond=Person("Raymond","@raymondh")>>># Python 3.7>>>timeit("raymond.twitter",globals=globals())0.05876131607996285>>># Python 3.8>>>timeit("raymond.twitter",globals=globals())0.0377705999400132

>>>importsys>>># Python 3.7>>>sys.getsizeof(list(range(20191014)))181719232>>># Python 3.8>>>sys.getsizeof(list(range(20191014)))161528168

So, Should You Upgrade to Python 3.8?

Let’s start with the simple answer. If you want to try out any of the new features you have seen here, then you do need to be able to use Python 3.8. Tools likepyenv andAnaconda make it easy to have several versions of Python installed side by side. Alternatively, you can run theofficial Python 3.8 Docker container. There is no downside to trying out Python 3.8 for yourself.

Now, for the more complicated questions. Should you upgrade your production environment to Python 3.8? Should you make your own project dependent on Python 3.8 to take advantage of the new features?

You should have very few issues running Python 3.7 code in Python 3.8. Upgrading your environment to run Python 3.8 is therefore quite safe, and you would be able to take advantage of theoptimizations made in the new version. Different beta-versions of Python 3.8 have already been available for months, so hopefully most bugs are already squashed. However, if you want to be conservative, you might hold out until the first maintenance release (Python 3.8.1) is available.

Once you’ve upgraded your environment, you can start to experiment with features that are only in Python 3.8, such asassignment expressions andpositional-only arguments. However, you should be conscious about whether other people depend on your code, as this will force them to upgrade their environment as well. Popular libraries will probably mostly support at least Python 3.6 for quite a while longer.

SeePorting to Python 3.8 for more information about preparing your code for Python 3.8.