338

Because I am used to the old ways of duck typing in Python, I fail to understand the need for ABC (abstract base classes). Thehelp is good on how to use them.

I tried to read the rationale in thePEP, but it went over my head. If I was looking for a mutable sequence container, I would check for__setitem__, or more likely try to use it (EAFP). I haven't come across a real life use for thenumbers module, which does use ABCs, but that is the closest I have to understanding.

Can anyone explain the rationale to me, please?

mc_kaiser's user avatar
mc_kaiser
7578 silver badges18 bronze badges
askedAug 25, 2010 at 22:43
Muhammad Alkarouri's user avatar

6 Answers6

349

@Oddthinking's answer is not wrong, but I think it misses thereal,practical reason Python has ABCs in a world of duck-typing.

Abstract methods are neat, but in my opinion they don't really fill any use-cases not already covered by duck typing. Abstract base classes' real power lies inthe way they allow you to customise the behaviour ofisinstance andissubclass. (__subclasshook__ is basically a friendlier API on top of Python's__instancecheck__ and__subclasscheck__ hooks.) Adapting built-in constructs to work on custom types is very much part of Python's philosophy.

Python's source code is exemplary.Here is howcollections.Container is defined in the standard library (at time of writing):

class Container(metaclass=ABCMeta):    __slots__ = ()    @abstractmethod    def __contains__(self, x):        return False    @classmethod    def __subclasshook__(cls, C):        if cls is Container:            if any("__contains__" in B.__dict__ for B in C.__mro__):                return True        return NotImplemented

This definition of__subclasshook__ says that any class with a__contains__ attribute is considered to be a subclass of Container, even if it doesn't subclass it directly. So I can write this:

class ContainAllTheThings(object):    def __contains__(self, item):        return True>>> issubclass(ContainAllTheThings, collections.Container)True>>> isinstance(ContainAllTheThings(), collections.Container)True

In other words,if you implement the right interface, you're a subclass! ABCs provide a formal way to define interfaces in Python, while staying true to the spirit of duck-typing. Besides, this works in a way that honours theOpen-Closed Principle.

Python's object model looks superficially similar to that of a more "traditional" OO system (by which I mean Java*) - we got yer classes, yer objects, yer methods - but when you scratch the surface you'll find something far richer and more flexible. Likewise, Python's notion of abstract base classes may be recognisable to a Java developer, but in practice they are intended for a very different purpose.

I sometimes find myself writing polymorphic functions that can act on a single item or a collection of items, and I findisinstance(x, collections.Iterable) to be much more readable thanhasattr(x, '__iter__') or an equivalenttry...except block. (If you didn't know Python, which of those three would make the intention of the code clearest?)

That said, I find that I rarely need to write my own ABC and I typically discover the need for one through refactoring. If I see a polymorphic function making a lot of attribute checks, or lots of functions making the same attribute checks, that smell suggests the existence of an ABC waiting to be extracted.

*without getting into the debate over whether Java is a "traditional" OO system...


Addendum: Even though an abstract base class can override the behaviour ofisinstance andissubclass, it still doesn't enter theMRO of the virtual subclass. This is a potential pitfall for clients: not every object for whichisinstance(x, MyABC) == True has the methods defined onMyABC.

class MyABC(metaclass=abc.ABCMeta):    def abc_method(self):        pass    @classmethod    def __subclasshook__(cls, C):        return Trueclass C(object):    pass# typical client codec = C()if isinstance(c, MyABC):  # will be true    c.abc_method()  # raises AttributeError

Unfortunately this one of those "just don't do that" traps (of which Python has relatively few!): avoid defining ABCs with both a__subclasshook__ and non-abstract methods. Moreover, you should make your definition of__subclasshook__ consistent with the set of abstract methods your ABC defines.

answeredOct 11, 2013 at 22:20
Benjamin Hodgson's user avatar
Sign up to request clarification or add additional context in comments.

7 Comments

"if you implement the right interface, you're a subclass" Thanks a lot for that. I don't know if Oddthinking missed it, but I certainly did. FWIW,isinstance(x, collections.Iterable) is clearer for me, and I do know Python.
Excellent post. Thank you. I think that the Addendum, "just don't do that" trap, is a bit like doing normal subclass inheritance but then having theC subclass delete (or screw up beyond repair) theabc_method() inherited fromMyABC. The principal difference is that it's the superclass that is screwing up the inheritance contract, not the subclass.
Would you not have to doContainer.register(ContainAllTheThings) for the given example to work?
@BoZenKhaa The code in the answer works! Try it! The meaning of__subclasshook__ is "any class which satisfies this predicate is considered a subclass for the purposes ofisinstance andissubclass checks,regardless of whether it was registered with the ABC, andregardless of whether it's a direct subclass". As I said in the answer,if you implement the right interface, you're a subclass!
Yes, of course it does, I was just being blind to a typo in my code, thank you :-). Great post. I should get some IDE.
|
214

Short version

ABCs offer a higher level of semantic contract between clients and the implemented classes.

Long version

There is a contract between a class and its callers. The class promises to do certain things and have certain properties.

There are different levels to the contract.

At a very low level, the contract might include the name of a method or its number of parameters.

In a staticly-typed language, that contract would actually be enforced by the compiler. In Python, you can useEAFP or type introspection to confirm that the unknown object meets this expected contract.

But there are also higher-level, semantic promises in the contract.

For example, if there is a__str__() method, it is expected to return a string representation of the object. Itcould delete all contents of the object, commit the transaction and spit a blank page out of the printer... but there is a common understanding of what it should do, described in the Python manual.

That's a special case, where the semantic contract is described in the manual. What should theprint() method do? Should it write the object to a printer or a line to the screen, or something else? It depends - you need to read the comments to understand the full contract here. A piece of client code that simply checks that theprint() method exists has confirmed part of the contract - that a method call can be made, but not that there is agreement on the higher level semantics of the call.

Defining an Abstract Base Class (ABC) is a way of producing a contract between the class implementers and the callers. It isn't just a list of method names, but a shared understanding of what those methods should do. If you inherit from this ABC, you are promising to follow all the rules described in the comments, including the semantics of theprint() method.

Python's duck-typing has many advantages in flexibility over static-typing, but it doesn't solve all the problems. ABCs offer an intermediate solution between the free-form of Python and the bondage-and-discipline of a staticly-typed language.

BeeOnRope's user avatar
BeeOnRope
66.1k20 gold badges243 silver badges445 bronze badges
answeredAug 26, 2010 at 3:59
Oddthinking's user avatar

10 Comments

I think that you have a point there, but I can't follow you. So what is the difference, in terms of contract, between a class that implements__contains__ and a class that inherits fromcollections.Container? In your example, in Python there was always a shared understanding of__str__. Implementing__str__ makes the same promises as inheriting from some ABC and then implementing__str__. In both cases you can break the contract; there are no provable semantics such as the ones we have in static typing.
collections.Container is a degenerate case, that only includes\_\_contains\_\_, and only to mean the predefined convention. Using an ABC doesn't add much value by itself, I agree. I suspect it was added to allow (for e.g.)Set to inherit from it. By the time you get toSet, suddenly belonging to the the ABC has considerable semantics. An item can't belong to the collection twice. That is NOT detectable by the existence of methods.
Yes, I thinkSet is a better example thanprint(). I was attempting to find a method name whose meaning was ambiguous, and couldn't be grokked by the name alone, so you couldn't be sure that it would do the right thing just by its name and the Python manual.
Any chance of rewriting the answer withSet as the example instead ofprint?Set makes a lot of sense, @Oddthinking.
I think this article explains it very well:dbader.org/blog/abstract-base-classes-in-python
|
158

A handy feature of ABCs is that if you don't implement all necessary methods (and properties) you get an error upon instantiation, rather than anAttributeError, potentially much later, when you actually try to use the missing method.

from abc import ABCMeta, abstractmethod# python2class Base(object):    __metaclass__ = ABCMeta    @abstractmethod    def foo(self):        pass    @abstractmethod    def bar(self):        pass# python3class Base(object, metaclass=ABCMeta):    @abstractmethod    def foo(self):        pass    @abstractmethod    def bar(self):        passclass Concrete(Base):    def foo(self):        pass    # We forget to declare `bar`c = Concrete()# TypeError: "Can't instantiate abstract class Concrete with abstract methods bar"

Example fromhttps://dbader.org/blog/abstract-base-classes-in-python

Edit: to include python3 syntax, thanks @PandasRocks

answeredMay 19, 2015 at 14:46
cerberos's user avatar

3 Comments

if Base is defined in a separate file, you must inherit from it as Base.Base or change the import line to 'from Base import Base'
It should be noted that in Python3 the syntax is slightly different. See this answer:stackoverflow.com/questions/28688784/…
Coming from a C# background, this is thethe reason to use abstract classes. You're providing functionality, but stating that the functionality requires further implementation. The other answers seem to miss this point.
27

It will make determining whether an object supports a given protocol without having to check for presence of all the methods in the protocol or without triggering an exception deep in "enemy" territory due to non-support much easier.

answeredAug 25, 2010 at 22:58
Ignacio Vazquez-Abrams's user avatar

Comments

16

Abstract methods make sure that whatever method you are calling in the parent class, has to also appear in the child class. Below are normal ways of calling and using abstract.The program is written in python3

Normal way of calling:

class Parent:    def method_one(self):        raise NotImplemented()    def method_two(self):        raise NotImplementedError()class Son(Parent):   def method_one(self):       return 'method_one() is called'c = Son()c.method_one()

'method_one() is called'

c.method_two()

NotImplementedError

With Abstract method:

from abc import ABCMeta, abstractmethodclass Parent(metaclass=ABCMeta):    @abstractmethod    def method_one(self):        raise NotImplementedError()    @abstractmethod    def method_two(self):        raise NotImplementedError()class Son(Parent):    def method_one(self):        return 'method_one() is called'c = Son()

TypeError: Can't instantiate abstract class Son with abstract methods method_two.

Since method_two is not called in child class we got error. The proper implementation is below:

from abc import ABCMeta, abstractmethodclass Parent(metaclass=ABCMeta):    @abstractmethod    def method_one(self):        raise NotImplementedError()    @abstractmethod    def method_two(self):        raise NotImplementedError()class Son(Parent):    def method_one(self):        return 'method_one() is called'    def method_two(self):        return 'method_two() is called'c = Son()c.method_one()

'method_one() is called'

Arindam Roychowdhury's user avatar
Arindam Roychowdhury
6,6015 gold badges59 silver badges66 bronze badges
answeredJul 19, 2018 at 6:48

3 Comments

Thanks. Isn't that the same point made in cerberos answer above?
I think this answer is better since it also shows you what happen if you are not using theabstractmethod.
You mean all function defined with@abstractmethod decorator must be implemented in the child class. If we leave a single one, we will get theCan't instantiate abstract class error. If this is true, than this is very useful thing.
-1

ABC's enable design patterns and frameworks to be created. Please see this pycon talk by Brandon Rhodes:

Python Design Patterns 1

The protocols within Python itself (not to mention iterators, decorators, and slots (which themselves implement the FlyWeight pattern)) are all possible because of ABC's (albeit implemented as virtual methods/classes in CPython).

Duck typing does make some patterns trivial in python, which Brandon mentions, but many other patterns continue to pop up and be useful in Python, e.g. Adapters.

In short, ABC's enable you to write scalable and reusable code. Per the GoF:

  1. Program to an interface, not an implementation (inheritance breaks encapsulation; programming to an interface promotes loose-coupling/inversion of control/the "HollyWood Principle: Don't call us, we'll call you")

  2. Favor object composition over class inheritance (delegate the work)

  3. Encapsulate the concept that varies (the open-closed principle makes classes open for extension, but closed for modification)

Additionally, with the emergence of static type checkers for Python (e.g.mypy), an ABC can be used as a type instead ofUnion[...] for every object a function accepts as an argument or returns. Imagine having to update the types, not the implementation, every time your code base supports a new object? That gets unmaintainable (doesn't scale) very fast.

answeredNov 6, 2021 at 23:33
adam.hendry's user avatar

2 Comments

In short, ABCs effectively do nothing except formal verification, and are rather like a standardized comment or type hint. Note that you can also program to an interface without that interface being formally declared in an ABC. ABCs themselves don't enable you to do anything, you can as well do 1-3 with just plain classes, so they are just a formality on top.
1-3 are just general and modern advice on inheritance, not specifically related to why Python needs ABC to do any of that.

Your Answer

Sign up orlog in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

By clicking “Post Your Answer”, you agree to ourterms of service and acknowledge you have read ourprivacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.