Browse Topics Guided Learning Paths
Basics Intermediate Advanced

api best-practices career community databases data-science data-structures data-viz devops django docker editors flask front-end gamedev gui machine-learning numpy projects python testing tools web-dev web-scraping

Python Protocols: Leveraging Structural Subtyping

byLeodanis Pozo Ramos Jul 17, 2024intermediate python

Table of Contents

Remove ads

In Python, aprotocol specifies the methods and attributes that a class must implement to be considered of a given type. Protocols are important in Python’stype hint system, which allows for static type checking through external tools, such asmypy,Pyright, andPyre.

Before there were protocols, these tools could only check fornominal subtyping based on inheritance. There was no way to check forstructural subtyping, which relies on the internal structure of classes. This limitation affected Python’sduck typing system, which allows you to use objects without considering their nominal types. Protocols overcome this limitation, making static duck typing possible.

In this tutorial, you’ll:

Gain clarity around the use of the termprotocol in Python
Learn howtype hints facilitatestatic type checking
Learn how protocols allowstatic duck typing
Create custom protocols with theProtocol class
Understand the differences betweenprotocols andabstract base classes

To get the most out of this tutorial, you’ll need to know the basics ofobject-oriented programming in Python, including concepts such asclasses andinheritance. You should also know abouttype checking andduck typing in Python.

Get Your Code:Click here to download the free sample code that shows you how to leverage structural subtyping with Python protocols

Take the Quiz: Test your knowledge with our interactive “Python Protocols: Leveraging Structural Subtyping” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Protocols: Leveraging Structural Subtyping

Take this quiz to test your understanding of how to create and use Python protocols while providing type hints for your functions, variables, classes, and methods.

The Meaning of “Protocol” in Python

During Python’s evolution, the termprotocol became overloaded with two subtly different meanings. The first meaning refers to internal protocols, such as theiterator,context manager, anddescriptor protocols.

These protocols are widely understood in the community and consist ofspecial methods that make up a given protocol. For example, the.__iter__() and.__next__() methods define the iterator protocol.

Python 3.8 introduced a second, slightly different type ofprotocol. These protocols specify the methods and attributes that a class must implement to be considered of a given type. So, these protocols also have to do with a class’s internal structure.

With this kind of protocol, you can define interchangeable classes as long as they share a common internal structure. This feature allows you to enforce a relationship between types or classes without the burden ofinheritance. This relationship is known asstructural subtyping orstatic duck typing.

In this tutorial, you’ll focus on this second meaning of the term protocol. First, you’ll have a look at how Python manages types.

Remove ads

Dynamic and Static Typing in Python

Python is a dynamically typed language, which means that the Python interpreter checks an object’s type when the code runs. It also means that while a variable can only reference one object at a time, the type of that object can change during the variable’s lifetime.

For example, you can have a variable that starts as a string and changes into an integer number:

Python

>>>value="One hundred">>>value'One hundred'>>>value=100>>>value100

In this example, you have a variable that starts as a string. Later in your code, you change the variable’s value to an integer.

Because of its dynamic nature, Python has embraced a flexible typing system that’s known as duck typing.

Duck Typing

Duck typing is atype system in which an object is considered compatible with a given type if it has all themethods andattributes that the type requires. This typing system supports the ability to use objects of independent and decoupled classes in a specific context as long as they adhere to some common interface.

Note: To dive deeper into duck typing, check out theDuck Typing in Python: Writing Flexible and Decoupled Code tutorial.

As an example of duck typing, you can consider built-in container data types, such aslists,tuples,strings,dictionaries, andsets. All of these data types support iteration:

Python

>>>numbers=[1,2,3]>>>person=("Jane",25,"Python Dev")>>>letters="abc">>>ordinals={"one":"first","two":"second","three":"third"}>>>even_digits={2,4,6,8}>>>containers=[numbers,person,letters,ordinals,even_digits]>>>forcontainerincontainers:...forelementincontainer:...print(element,end=" ")...print()...1 2 3Jane 25 Python Deva b cone two three8 2 4 6

In this code snippet, you define a few variables using different built-in types. Then, you start afor loop over the collections and iterate over each of them to print their elements to the screen. Even though the built-in types are significantly different from one another, they all support iteration.

The duck typing system allows you to create code that can work with different objects, provided that they share a common interface. This system allows you to set relationships between classes that don’t rely on inheritance, which produces flexible and decoupled code.

Type Hints and Type Checking

Even though Python is a dynamically typed language that widely relies on duck typing, not knowing the type of arguments,return values, and attributes can be a source of errors. This is especially true in large codebases where functions and classes are spread through severalmodules and packages.

To overcome the potential issues, Python 3.5 introducedtype hints or optional static typing. Type hints let you optionally specify types of arguments, return values, and attributes in your functions and classes. Then, you can check these types with astatic type checker, such as mypy, and get useful information that can help you debug and improve your code, making it more robust.

Here’s a quick overview of some great benefits of using type hints in your code:

Improved code readability: You can explicitly state the type of arguments, return values, and attributes, which enhances code readability for developers, making the interface of functions and classes easier to understand.
Support for static type checking: You can provide type hints in your code and then use tools likemypy,Pyright, andPyre to perform static type checking. These tools will allow you to find type inconsistencies and mismatches early in the development process so that you can fix them and make your code more robust.
Better development environments: You can use type hints and take advantage of a moderncode editor or IDE’s enhanced code completion, function signature information, and inline documentation, which improve your life quality as a Python developer. These features can also speed up development and reduce bugs caused by incorrect type usage.
Aids in refactoring: You can make refactors that involve changing the type of arguments, return values, and attributes of a given function or class. In this situation, the type checkers will help you find all the parts of the code that you must update to align with the new definitions.
Improved documentation: You can take advantage of type hints when writing your code’s documentation with tools likeMkDocs. This way, the type information in the documentation will stay in sync with the code.

Python doesn’t do anything with type hints at runtime. They are stored in an.__annotations__ attribute and otherwise ignored. So, apart from the improved code readability, you need external tools like astatic type checker or a modern code editor or IDE to enjoy these benefits.

Note: To learn more about type checking in Python, check out thePython Type Checking (Guide) tutorial.

As an example of using type hints, consider the following function that adds two numbers:

Pythoncalculations.py

 1fromtypingimportUnion 2 3defadd(a:Union[float,int],b:Union[float,int])->float: 4returnfloat(a+b) 5 6print(add(2,4)) 7 8print(add("2","4"))

Thetyping module plays a crucial role in defining type hints. It defines classes likeUnion that you can use to specify multiple possible types of variables, function arguments, and return values. In this example, you add type hints for the function’s arguments,a andb, and for the return value.

The arguments toadd() can be floating-point or integer numbers. To set up these type hints, you use theUnion class with thefloat andint types. Alternatively, you can use the pipe operator to express the same type hint. So, for example, you can write something likea: float | int.

Then, you use the arrow syntax (-> float) to state that the function’s return value will be a floating-point number. Finally, you have a couple of calls toadd(). Here’s what you get when you run this script:

Shell

$pythoncalculations.py6.024.0

The first result is correct because you used two numbers as arguments. Unfortunately, the second result is incorrect, but your code didn’t break. It ran without apparent problems. How would you avoid this type of issue? You can run some static type checking in your code.

To perform static type checking on the above code, you need an external tool. For example, you can use mypy, but first, you need to install it with the following command:

Shell

$python-mpipinstallmypy

This command will install the mypy static type checker from PyPI. Now you can run the following command against yourcalculations.py file:

Shell

$mypycalculations.pycalculations.py:8: error: Argument 1 to "add" has incompatible type "str";    expected "float | int"  [arg-type]calculations.py:8: error: Argument 2 to "add" has incompatible type "str";    expected "float | int"  [arg-type]Found 2 errors in 1 file (checked 1 source file)

With this command, you run a static type analysis on your code. As a result, you learn that your code has two errors in line 8. Those errors warn you about type inconsistencies in your code. With this information, you can go to your code and fix the inconsistencies so that the code works correctly.

Remove ads

Duck Typing vs Type Hints

As you’ve learned, duck typing allows for flexible and dynamic code. With duck typing, you can use different and unrelated objects in a given context if those objects have the expected methods and attributes. You don’t have to ensure that the objects share a common parent type through inheritance.

When you add type hints to a piece of code that relies on duck typing, you can encounter some challenges. Here’s a toy example that illustrates how type hints and duck typing can collide:

Pythonbirds_v1.py

 1classDuck: 2defquack(self): 3return"The duck is quacking!" 4 5defmake_it_quack(duck:Duck)->str: 6returnduck.quack() 7 8classPerson: 9defquack(self):10return"The person is imitating a duck quacking!"1112print(make_it_quack(Duck()))1314print(make_it_quack(Person()))

In this code, you define aDuck class with a.quack() method. Then, you define themake_it_quack() function that takes an argument of typeDuck. Next, you create aPerson class to use in a duck typing context. In this example, the calls toprint() will work. However, in the second call, you have a type inconsistency because an instance ofPerson isn’t of typeDuck.

Here’s what you get when you run mypy on this file:

Shell

$mypybirds_v1.pybirds_v1.py:14: error: Argument 1 to "make_it_quack" has incompatible type    "Person"; expected "Duck"  [arg-type]Found 1 error in 1 file (checked 1 source file)

The type checker displays an error telling you that the call toprint() in line 14 is using an object with an incompatible type.

One way you can fix this issue is to use inheritance:

Pythonbirds_v2.py

classQuackingThing:defquack(self):raiseNotImplementedError("Subclasses must implement this method")classDuck(QuackingThing):defquack(self):return"The duck is quacking!"classPerson(QuackingThing):defquack(self):return"The person is imitating a duck quacking!"defmake_it_quack(duck:QuackingThing)->str:returnduck.quack()print(make_it_quack(Duck()))print(make_it_quack(Person()))

In this new implementation, you create a new class calledQuackingThing and use it as the parent class forDuck andPerson. Now, the type checker passes:

Shell

$mypybirds_v2.pySuccess: no issues found in 1 source file

The type checker’s output is clean. However, you traded duck typing for inheritance. Now, your classes are tightly coupled, even if they don’t have a clear inheritance relationship. How can you fix this collision between duck typing and type hints? This is whereprotocols come into the scene.

Structural Subtyping and Protocols

In Python’s type system, you’ll find two ways to decide whether two objects are compatible as types:

Nominal subtyping is strictly based on inheritance. A class that inherits from a parent class is a subtype of its parent.
Structural subtyping is based on the internal structure of classes. Two classes with the same methods and attributes are structural subtypes of one another.

To understand the first subtyping strategy, say you have a class hierarchy to represent animals:

Pythonanimals_v1.py

classAnimal:def__init__(self,name):self.name=namedefeat(self):print(f"{self.name} is eating.")defdrink(self):print(f"{self.name} is drinking.")classDog(Animal):defbark(self):print(f"{self.name} is barking.")classCat(Animal):defmeow(self):print(f"{self.name} is meowing.")

In this code,Dog andCat are nominal subtypes ofAnimal. This means that you can use instances ofDog orCat whenAnimal instances are expected. This form of subtyping is quick to understand and matches how the built-inisinstance() works:

Python

>>>fromanimals_v1importAnimal,Cat,Dog>>>pluto=Dog("Pluto")>>>isinstance(pluto,Animal)True>>>tom=Cat("Tom")>>>isinstance(tom,Animal)True

The built-inisinstance() function allows you to check whether a given object is an instance of a class. This function considers subtypes when running the check. That’s why you getTrue in both examples above.

Checking types explicitly isn’t a popular practice in Python. Instead, the language favors duck typing, where you use the objects without considering their types but only the expected operations. Here’s where structural subtyping comes in handy.

For example, you can reimplement the animal classes without setting a formal relationship between them:

Pythonanimals_v2.py

classDog:def__init__(self,name):self.name=namedefeat(self):print(f"{self.name} is eating.")defdrink(self):print(f"{self.name} is drinking.")defmake_sound(self):print(f"{self.name} is barking.")classCat:def__init__(self,name):self.name=namedefeat(self):print(f"{self.name} is eating.")defdrink(self):print(f"{self.name} is drinking.")defmake_sound(self):print(f"{self.name} is meowing.")

NowDog andCat don’t have a strict inheritance relationship. They’re completely decoupled and independent classes.

However, they implement the samepublic interface. In other words, they have the same internal structure, including methods and attributes. Because of this characteristic, they’re structural subtypes, and you can use them in a duck typing context:

Python

>>>fromanimals_v2importCat,Dog>>>foranimalin[Cat("Tom"),Dog("Pluto")]:...animal.eat()...animal.drink()...animal.make_sound()...Tom is eating.Tom is drinking.Tom is meowing.Pluto is eating.Pluto is drinking.Pluto is barking.

These classes work okay in a duck typing context. However, up to this point, you don’t have a formal way to indicate thatCat andDog are subtypes. You only know they are because the code tells you they have the same internal structure. To formalize this subtype relationship, you can use a protocol.

As you’ve already learned,protocols allow you to specify the expected methods and attributes that a class should have to support a given feature without requiring explicit inheritance. So, protocols are explicit sets of methods and attributes.

In practice, a class can support multiple protocols. For example, you can informally say that yourDog andCat classes have aliving protocol consisting of the.eat() and.drink() methods, which are the operations required to support life. You can also say that the classes have asounding protocol comprised of the.make_sound() method.

Note: In Python’s built-in classes, you’ll find many examples of classes that support multiple protocols. For example, lists, tuples, and strings support thesequence anditerable protocols.

With this in mind, you can create other classes that support any of these protocols:

Pythonperson.py

classPerson:def__init__(self,name):self.name=namedefeat(self):print(f"{self.name} is eating.")defdrink(self):print(f"{self.name} is drinking.")deftalk(self):print(f"{self.name} is talking.")

ThisPerson class supports theliving protocol because it implements the required behaviors,.eat() and.drink(). However, the class doesn’t implement the.make_sound() method, so it doesn’t support thesounding protocol.

Protocols are particularly useful when it’s impractical to modify the inheritance structure of classes. With protocols, you can focus on defining the desired behavior and characteristics without having to design complex inheritance relationships.

Remove ads

Static Duck Typing With Protocols

As you’ve already learned, duck typing and type hints collide, imposing restrictions on Python coders who want to use both techniques. Python3.8 introduced a way to create formal protocols in thetyping system.

From this version on, thetyping module defines a base class calledProtocol that you can use to create custom protocols. This new class provides a formal way to support structural subtyping in the type hint system.

Since Python 3.8, the static type checkers started to support structural subtyping, which means that they check whether objects are of a specific nominal type and also whether they meet the requirement to be of a given structural type.

Consider the following toy example:

Pythonadder_v1.py

fromtypingimportProtocolclassAdder(Protocol):defadd(self,x,y):...classIntAdder:defadd(self,x,y):returnx+yclassFloatAdder:defadd(self,x,y):returnx+ydefadd(adder:Adder)->None:print(adder.add(2,3))add(IntAdder())add(FloatAdder())

In this code, you define a class calledAdder by inheriting fromtyping.Protocol.Adder implements an.add() method, which defines theAdder protocol itself. Note that protocol methods don’t have a body, which you typically indicate with theellipsis (...) syntax.

Then, you define two classes,IntAdder andFloatAdder. These classes implement theAdder protocol because they have an.add() method. Therefore, you can use objects of either class as arguments to theadd() function, which takes anAdder object as an argument.

The static type checker will be happy with your type hints on theadd() function:

Shell

$mypyadder_v1.pySuccess: no issues found in 1 source file

The static type check passes because bothIntAdder andFloatAdder support theAdder protocol. In this case, mypy considers the object’s internal structure rather than its nominal type.

Note that you haven’t made these classes inherit fromAdder. They’re completely independent and decoupled classes that you can use in a duck typing context, such as theadd() function.

Custom Protocols in Python

There’s much more to learn about creating custom protocols in Python. For example, you can create protocols through single or multiple inheritance. You can have protocols with different types of members. You can even create generic protocols.

In the following sections, you’ll learn about these topics and how they can help you improve your custom protocols and make them meet your typing requirements.

Creating Custom Protocols

You’ve already learned that you can define custom protocols using theProtocol class from thetyping module. Adding type hints to the equation will make your type checking process more strict.

Consider the following example:

Pythonadder_v2.py

fromtypingimportProtocolclassAdder(Protocol):defadd(self,x:float,y:float)->float:...classIntAdder:defadd(self,x,y):returnx+ydefadd(adder:Adder)->None:print(adder.add(2,3))add(IntAdder())# mypy: Success: no issues found in 1 source file

Because theIntAdder class doesn’t use explicit type hints, mypy considers the method’s arguments and return value to be of typeAny, which makes the type check succeed regardless of the types used.

Now, go ahead and add type hints for the arguments to.add() and its return value in theIntAdder class to enable mypy to perform type checking:

Pythonadder_v2.py

# ...classIntAdder:defadd(self,x:int,y:int)->int:returnx+ydefadd(adder:Adder)->None:print(adder.add(2,3))add(IntAdder())

In this update, you’ve stated that the.add() method will take integer arguments and return an integer value. Go ahead and run mypy on this new version ofadder_v2.py:

Shell

$mypyadder_v2.pyadder_v2.py:17: error: Argument 1 to "add" has incompatible type "IntAdder";    expected "Adder"  [arg-type]adder_v2.py:17: note: Following member(s) of "IntAdder" have conflicts:adder_v2.py:17: note:     Expected:adder_v2.py:17: note:         def add(self, x: float, y: float) -> floatadder_v2.py:17: note:     Got:adder_v2.py:17: note:         def add(self, x: int, y: int) -> intFound 1 error in 1 file (checked 1 source file)

This output tells you that yourIntAdder class is incompatible with theAdder protocol because.add() takes and returns incompatible types. How do you make the check pass? You could update theAdder protocol to accept either integer or floating-point numbers:

Pythonadder_v3.py

fromtypingimportProtocolclassAdder(Protocol):defadd(self,x:int|float,y:int|float)->int|float:...# ...

In this case, the.add() method can take either integer or float arguments. It can also return arguments of either type.

Go ahead and run mypy again:

Shell

$mypyadder_v3.pyadder_v2.py:17: error: Argument 1 to "add" has incompatible type "IntAdder";    expected "Adder"  [arg-type]adder_v2.py:17: note: Following member(s) of "IntAdder" have conflicts:adder_v2.py:17: note:     Expected:adder_v2.py:17: note:         def add(self, x: int | float, y: int | float)    -> int | floatadder_v2.py:17: note:     Got:adder_v2.py:17: note:         def add(self, x: int, y: int) -> intFound 1 error in 1 file (checked 1 source file)

This update doesn’t solve the issue. You need to take another approach, and that’s when generic protocols come in handy.

Remove ads

Building Generic Protocols

Protocol classes can be generic. To create a generic protocol, you can use thetyping.TypeVar class. The syntax for a generic protocol is like the following:

Python

fromtypingimportProtocol,TypeVarT=TypeVar("T")classGenericProtocol(Protocol[T]):defmethod(self,arg:T)->T:...

In this code snippet, you first define a generic type,T, using theTypeVar class. Then, you create a generic protocol usingProtocol[T] as the parent class. You can also use generic types in the arguments and return values of the protocol’s methods.

You can use a generic protocol to overcome the issue you found in the previous section:

Pythonadder_v4.py

fromtypingimportProtocol,TypeVarT=TypeVar("T",bound=int|float)classAdder(Protocol[T]):defadd(self,x:T,y:T)->T:...classIntAdder:defadd(self,x:int,y:int)->int:returnx+yclassFloatAdder:defadd(self,x:float,y:float)->float:returnx+ydefadd(adder:Adder)->None:print(adder.add(2,3))add(IntAdder())add(FloatAdder())

In this example, you first define a generic type for your protocol. You use thebound argument to state that the generic type can be anint orfloat object. Then, you have your concrete adders. In this case, you haveIntAdder andFloatAdder to sum numbers.

If you’re using Python3.12, then you can use a simplified syntax:

Python

fromtypingimportProtocolclassAdder(Protocol):defadd[T:int|float](self,x:T,y:T)->T:...# ...

With this syntax, you don’t have to importTypeVar and create the generic type beforehand. You just have to use square brackets after the method’s name, use the genericT type, and optionally bound some existing types.

Exploring Possible Protocol Members

Up to this point, you’ve written protocols with regular instance methods. However, protocols can have different types of members, including the following:

You should use theClassVar class to distinguish between class attributes and instance attributes. Here’s a demo protocol class that defines all the above members:

Pythonmembers.py

fromabcimportabstractmethodfromtypingimportClassVar,ProtocolclassProtocolMembersDemo(Protocol):class_attribute:ClassVar[int]instance_attribute:str=""definstance_method(self,arg:int)->str:...@classmethoddefclass_method(cls)->str:...@staticmethoddefstatic_method(arg:int)->str:...@propertydefproperty_name(self)->str:...@property_name.setterdefproperty_name(self,value:str)->None:...@abstractmethoddefabstract_method(self)->str:...

You useClassVar to define a class attribute. Then, you have an instance attribute with a default value, which is optional.

It’s important to note that instance attributes must be declared at the class level for the type checker to consider them a part of the protocol. Otherwise, if you define instance attributes inside instance methods, which is a common practice in Python, then you’ll get an error from your type checker.

Next up, you have different types of methods. In this example, none of the methods have a defined implementation. However, you can have protocol methods with default implementations. Finally, note that all the members of a protocol class can have type hints.

Creating Subprotocols

You can take advantage of existing protocols to create new ones using inheritance. For example, say that you’re creating a platform to publish blog posts and video posts. You want to use proper type hints in all your classes and functions, so you write the following protocols:

Pythoncontents.py

fromtypingimportProtocolclassContentCreator(Protocol):defcreate_content(self)->str:...classBlogger(ContentCreator,Protocol):posts:list[str]defadd_post(self,title:str,content:str)->None:...classVlogger(ContentCreator,Protocol):videos:list[str]defadd_video(self,title:str,path:str)->None:...

In this code snippet, you have a base protocol calledContentCreator that defines a.create_content() method. Then, you create two derived protocols,Blogger andVlogger. These classes inherit from the base protocol,ContentCreator, which makes them subprotocols.

The classes also inherit fromProtocol. Note that this isn’t mandatory. However, it’s a best practice to ensure that the semantics and intentions are clear.

Next, you define two concrete classes:

Pythoncontents.py

# ...classBlog:def__init__(self):self.posts=[]defcreate_content(self)->str:return"Creating a post."defadd_post(self,title:str,content:str)->None:self.posts.append(f"{title}:{content}")print(f"Post added:{title}")classVlog:def__init__(self):self.videos=[]defcreate_content(self)->str:return"Recording a video."defadd_video(self,title:str,path:str)->None:self.videos.append(f"{title}:{path}")print(f"Video added:{title}")

TheBlog class meets theBlogger protocol while theVlog class meets theVlogger protocol. Now, you can have client code to use these classes:

Pythoncontents.py

# ...defproduce_content(creator:ContentCreator):print(creator.create_content())defadd_post(blogger:Blogger,title:str,content:str):blogger.add_post(title,content)defadd_video(vlogger:Vlogger,title:str,path:str):vlogger.add_video(title,path)

The first function allows you to create a new piece of content. It can take any object that satisfies theContentCreator protocol. The other two functions allow you to add a new blog post and video. They can take objects that satisfy theBlogger andVlogger protocols, respectively.

Now, you can use mypy to check if your type hints work:

Shell

$mypycontents.pySuccess: no issues found in 1 source file

Great! The output is clean. Your subprotocol classes work as expected!

Remove ads

Recursive Protocols

You can also definerecursive protocols, which are protocols that reference themselves in their definition. To reference a protocol, you must provide its name as strings.

Recursive protocols are useful for representing self-referential data structures likelinked lists. Consider the following example:

Pythonlinked_list.py

fromtypingimportOptional,ProtocolclassLinkedListNode(Protocol):value:intnext_node:Optional["LinkedListNode"]def__str__(self)->str:returnf"{self.value} ->{self.next_node}"

In this code snippet, you define a protocol calledLinkedListNode to represent a node in a linked list. The class has two instance attributes:.value to hold the current node’s value and.next_node to hold the next node in the list.

Note: To reference a protocol within its definition, you must include its name as a string literal to avoid errors. That’s because you can’t refer to a type that isn’t fully defined yet. While this limitation will change in the future, for now, you can use a future import as an alternative:

Python

from__future__importannotationsfromtypingimportOptional,ProtocolclassLinkedListNode(Protocol):value:intnext_node:Optional[LinkedListNode]def__str__(self)->str:returnf"{self.value} ->{self.next_node}"

By importingannotations from__future__, you ensure that the annotations or type hints are treated as strings, allowing you to referenceLinkedListNode even before it’s fully defined.

The.next_node attribute should also be a node object, so you use a recursive reference to theLinkedListNode as a string. In this example, you use theOptional class to express that.next_node can also beNone.

Now, go ahead and create a concrete node and some client code:

Pythonlinked_list.py

# ...classNode:def__init__(self,value:int,next_node:Optional["LinkedListNode"]=None,):self.value=valueself.next_node=next_nodedef__str__(self)->str:returnf"{self.value} ->{self.next_node}"defprint_linked_list(start_node:LinkedListNode):print(start_node)node3=Node(3)node2=Node(2,node3)node1=Node(1,node2)print_linked_list(node1)

TheNode class satisfies theLinkedListNode protocol. Next, you have theprint_linked_list() function, which takes aLinkedListNode object as an argument and prints the linked list to the screen. Finally, you have code to create a few nodes and callprint_linked_list().

You can run mypy on this file now:

Shell

$mypylinked_list.pySuccess: no issues found in 1 source file

The output from mypy is clean, so your recursive protocol works as expected. You’ve written the correct type hints for your classes, which is great!

Predefined Protocols in Python

Python has several predefined protocols.Iterable andIterator are good examples. A few of them are in thetyping module under theProtocols section. However, most of the currently available protocols live in thecollections.abc module because they’re implemented as abstract base classes.

Some of the most common predefined protocols include the following:

Class	Methods
`Container`	`.__contains__()`
`Hashable`	`.__hash__()`
`Iterable`	`.__iter__()`
`Iterator`	`.__next__()` and`.__iter__()`
`Reversible`	`.__reversed__()`
`Generator`	`.send()`,`.throw()`,`.close()`,`.__iter__()`, and`.__next__()`
`Sized`	`.__len__()`
`Callable`	`.__call__()`
`Collection`	`.__contains__()`,`.__iter__()`, and`.__len__()`
`Sequence`	`.__getitem__()`,`.__len__()`,`.__contains__()`,`.__iter__()`,`.__reversed__()`,`.index()`, and`.count()`
`MutableSequence`	`.__getitem__()`,`.__setitem__()`,`.__delitem__()`,`.__len__()`,`.insert()`,`.append()`,`.clear()`,`.reverse()`,`.extend()`,`.pop()`,`.remove()`, and`.__iadd__()`
`ByteString`	`.__getitem__()` and`.__len__()`
`Set`	`.__contains__()`,`.__iter__()`,`.__len__()`,`.__le__()`,`.__lt__()`,`.__eq__()`,`.__ne__()`,`.__gt__()`,`.__ge__()`,`.__and__()`,`.__or__()`,`.__sub__()`,`.__xor__()`, and`.isdisjoint()`
`MutableSet`	`.__contains__()`,`.__iter__()`,`.__len__()`,`.add()`,`.discard()`,`.clear()`,`.pop()`,`.remove()`,`.__ior__()`,`.__iand__()`,`.__ixor__()`, and`.__isub__()`
`Mapping`	`.__getitem__()`,`.__iter__()`,`.__len__()`,`.__contains__()`,`.keys()`,`.items()`,`.values()`,`.get()`,`.__eq__()`, and`.__ne__()`
`MutableMapping`	`.__getitem__()`,`.__setitem__()`,`.__delitem__()`,`.__iter__()`,`.__len__()`,`.pop()`,`.popitem()`,`.clear()`,`.update()`, and`.setdefault()`
`AsyncIterable`	`.__aiter__()`
`AsyncIterator`	`.__anext__()` and`.__aiter__()`
`AsyncGenerator`	`.asend()`,`.athrow()`,`.aclose()`,`.__aiter__()`, and`.__anext__()`
`Buffer`	`.__buffer__()`

Even though these classes areabstract base classes (ABC) rather than formal protocols, you can use them as protocols in your type hints. The static type checkers should be able to process them as expected.

In some situations, you probably won’t need to use these classes because you can just use the concrete built-in type. For example, instead of using theSet ABC to type hint an argument or a return value, you can use the built-inset class.

However, there are situations where using the concrete type won’t meet your needs. For example, say that you want to code a function that takes some integer values and filters the even numbers, returning a list. Here’s the function without type hints:

Pythoneven_v1.py

deffilter_even_numbers(numbers):return[numberfornumberinnumbersifnumber%2==0]print(filter_even_numbers([1,2,3,4,5,6,7,8,9,10]))print(filter_even_numbers((1,2,3,4,5,6,7,8,9,10)))print(filter_even_numbers({1,2,3,4,5,6,7,8,9,10}))

This function works with any iterable object. In other words, you can use a list, tuple, set, or any other iterable object as an argument tofilter_even_numbers().

How would you type hint thenumbers argument in this function? You can think of doing something like the following:

Pythoneven_v2.py

deffilter_even_numbers(numbers:list[int])->list[int]:return[numberfornumberinnumbersifnumber%2==0]print(filter_even_numbers([1,2,3,4,5,6,7,8,9,10]))print(filter_even_numbers((1,2,3,4,5,6,7,8,9,10)))print(filter_even_numbers({1,2,3,4,5,6,7,8,9,10}))

Thelist[int] type hint fornumbers doesn’t work well. It causesnumbers to only accept list objects, which makes the function less generic. Your static type checker will fail with tuples, sets, or any other iterable:

Shell

$mypyeven_v2.pyeven_v2.py:5: error: Argument 1 to "filter_even_numbers" has incompatible    type "tuple[int, int, int, int, int, int, int, int, int, int]";    expected "list[int]"  [arg-type]even_v2.py:6: error: Argument 1 to "filter_even_numbers" has incompatible    type "set[int]"; expected "list[int]"  [arg-type]Found 2 errors in 1 file (checked 1 source file)

This output shows that the two final calls toprint() cause issues related to the data types of the input values for thenumbers argument.

To fix the issues and provide a generic type hint that allows your function to take different iterables of numbers, you can use theIterable ABC as in the code below:

Pythoneven_v3.py

fromcollections.abcimportIterabledeffilter_even_numbers(numbers:Iterable[int])->list[int]:return[numberfornumberinnumbersifnumber%2==0]print(filter_even_numbers([1,2,3,4,5,6,7,8,9,10]))print(filter_even_numbers((1,2,3,4,5,6,7,8,9,10)))print(filter_even_numbers({1,2,3,4,5,6,7,8,9,10}))

In this update, you import theIterable class fromcollections.abc. This class implements the.__iter__() method, satisfying the iterable protocol. Being an iterable of integer values, this method is exactly what yourfilter_even_numbers() function needs to work properly.

Now, go ahead and run mypy again:

Shell

$mypyeven_v3.pySuccess: no issues found in 1 source file

The output is now clean. This means that the type hints for the function work correctly. You can use this function with any input iterable of integer values.

Remove ads

Protocols vs Abstract Base Classes

Anabstract base class (ABC) is designed to be subclassed but not instantiated. This type of class defines a specific public interface (API) and enforces that interface in its subclasses. To create an abstract base class, you can use theABC class from theabc module.

To illustrate how you’ll typically use ABCs, consider the following example:

Pythonshapes_v1.py

fromabcimportABC,abstractmethodfrommathimportpiclassShape(ABC):@abstractmethoddefget_area(self)->float:pass@abstractmethoddefget_perimeter(self)->float:passclassCircle(Shape):def__init__(self,radius)->None:self.radius=radiusdefget_area(self)->float:returnpi*self.radius**2defget_perimeter(self)->float:return2*pi*self.radiusclassSquare(Shape):def__init__(self,side)->None:self.side=sidedefget_area(self)->float:returnself.side**2defget_perimeter(self)->float:return4*self.side

In this example, theShape class is an abstract base class. With this class, you’re stating that any class that you create as aShape subclass must have the.get_area() and.get_perimeter() methods in their public interface. That’s what theCircle andSquare classes do. If you fail to implement one of the required methods, then you’ll get an error.

You can use theShape class to provide type hints to your client code. Go ahead and add the following code to yourshapes_v1.py file:

Pythonshapes_v1.py

# ...defprint_shape_info(shape:Shape):print(f"Area:{shape.get_area()}")print(f"Perimeter:{shape.get_perimeter()}")circle=Circle(10)square=Square(5)print_shape_info(circle)print_shape_info(square)

The type hint for theshape argument will work correctly becauseCircle andSquare are subclasses ofShape.

In this example, you’ve used an ABC and inheritance to enforce a specific interface in a group of classes. However, sometimes it’s desirable to have a similar result without inheritance. That’s when you can use a protocol:

Pythonshapes_v2.py

frommathimportpifromtypingimportProtocolclassShape(Protocol):defget_area(self)->float:...defget_perimeter(self)->float:...classCircle:def__init__(self,radius)->None:self.radius=radiusdefget_area(self)->float:returnpi*self.radius**2defget_perimeter(self)->float:return2*pi*self.radiusclassSquare:def__init__(self,side)->None:self.side=sidedefget_area(self)->float:returnself.side**2defget_perimeter(self)->float:return4*self.sidedefprint_shape_info(shape:Shape):print(f"Area:{shape.get_area()}")print(f"Perimeter:{shape.get_perimeter()}")circle=Circle(10)square=Square(5)print_shape_info(circle)print_shape_info(square)

In this new implementation, you use a protocol instead of an ABC. Now, you don’t rely on inheritance for your type hints to work correctly. Your classes are decoupled from each other. Their only point of coincidence is that they have a piece of interface in common.

In short, the main difference between an abstract base class and a protocol is that the former works through a formal inheritance relationship, while the latter doesn’t need this relationship. Note that this difference doesn’t make ABCs better than protocols or vice versa. They have different use cases and purposes.

ABCs are suitable when you have control over the class hierarchy and want to define a consistent interface across subclasses. Meanwhile, protocols are useful in scenarios where modifying class hierarchies is impractical or when there’s no clear inheritance relationship between classes.

Considering Potential Downsides of Protocols

It’s worthwhile to mention a downside of protocols that ABCs don’t exhibit. For example, protocols can make the type checker accept a completely unrelated type, which happens to satisfy the protocol completely by accident but is otherwise inappropriate.

Here’s a relevant example:

Python

fromtypingimportProtocolclassMessage(Protocol):defencode(self)->bytes:...defsend(message:Message)->None:...send("Hello, World!")# Passes the type checker

In this example, strings happen to have the.encode() method, which returnsbytes and satisfies the protocol. However, you may want thesend() function to only accept instances of custom classes that implement theMessage protocol. This behavior creates a loophole in the type checking system.

Another potential downside of protocols is thatisinstance() will raise an exception when used with them. Consider the following code that takes advantage of the shapes example from the previous section:

Python

>>>fromshapes_v2importcircle,Shape>>>isinstance(circle,Shape)Traceback (most recent call last):...TypeError:Instance and class checks can only be used with    @runtime_checkable protocols

This check could be unreliable because protocols don’t set an inheritance relationship. If you need your classes to work withisinstance() even if they’re not subclasses, then you can use the@runtime_checkable decorator in the class definition:

Pythonshapes_v3.py

fromtypingimportProtocol,runtime_checkable@runtime_checkableclassShape(Protocol):...

The@runtime_checkable decorator marks a protocol class as a runtime protocol so that you can use it withisinstance() andissubclass().

Here’s how the previous example works now:

Python

>>>fromshapes_v3importcircle,Shape>>>isinstance(circle,Shape)True

After decorating yourShape protocol class with@runtime_checkable, the built-inisinstance() function works as you wanted.

Remove ads

Conclusion

Now you know how to use protocols in Python.Protocols let you define a type relationship between objects without the burden of inheritance. This relationship is based on the internal structure of classes.

With protocols, you can perform structural subtyping or static duck typing using Python’stype hint system and external static type checkers, like mypy, Pyright, and Pyre.

In this tutorial, you’ve:

Gained clarity about the different uses of the termprotocol in Python
Learned howtype hints facilitatestatic type checking
Learned how protocols provide support forstatic duck typing
Created custom protocols with theProtocol class
Understood the differences betweenprotocols andABCs

With this knowledge, you’re ready to get the most out of using Python’s type hint system and static type checkers.

Get Your Code:Click here to download the free sample code that shows you how to leverage structural subtyping with Python protocols

Interactive Quiz

Python Protocols: Leveraging Structural Subtyping

Take this quiz to test your understanding of how to create and use Python protocols while providing type hints for your functions, variables, classes, and methods.

🐍 Python Tricks 💌

Get a short & sweetPython Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

AboutLeodanis Pozo Ramos

Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites.

» More about Leodanis

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are: