Comprehensive Python Cheatsheet

^{Download text file,Fork me on GitHub,Check out FAQ orSwitch to dark theme.}

ToC = {'1. Collections': [List,Dictionary,Set,Tuple,Range,Enumerate,Iterator,Generator],'2. Types':       [Type,String,Regular_Exp,Format,Numbers,Combinatorics,Datetime],'3. Syntax':      [Function,Inline,Import,Decorator,Class,Duck_Type,Enum,Except],'4. System':      [Exit,Print,Input,Command_Line_Arguments,Open,Path,OS_Commands],'5. Data':        [JSON,Pickle,CSV,SQLite,Bytes,Struct,Array,Memory_View,Deque],'6. Advanced':    [Operator,Match_Stmt,Logging,Introspection,Threading,Coroutines],'7. Libraries':   [Progress_Bar,Plot,Table,Console_App,GUI,Scraping,Web,Profile],'8. Multimedia':  [NumPy,Image,Animation,Audio,Synthesizer,Pygame,Pandas,Plotly]}

#Main

if __name__ =='__main__':# Skips next line if file was imported.    main()# Runs `def main(): ...` function.

#List

<list> = [<el_1>, <el_2>, ...]# Creates new list. Also list(<collection>).

#Dictionary

<dict> = {key_1: val_1, key_2: val_2, ...}# Use `<dict>[key]` to get or set the value.

Counter

>>>from collectionsimport Counter>>>counter = Counter(['blue','blue','blue','red','red'])>>>counter['yellow'] +=1>>>print(counter.most_common())[('blue',3), ('red',2), ('yellow',1)]

#Set

<set> = {<el_1>, <el_2>, ...}# Use `set()` for empty set.

Frozen Set

• Is immutable and hashable.
• That means it can be used as a key in a dictionary or as an element in a set.

<frozenset> = frozenset(<collection>)

#Tuple

Tuple is an immutable and hashable list.

<tuple> = ()# Empty tuple.<tuple> = (<el>,)# Or: <el>,<tuple> = (<el_1>, <el_2> [, ...])# Or: <el_1>, <el_2> [, ...]

Named Tuple

Tuple's subclass with named elements.

>>>from collectionsimport namedtuple>>>Point = namedtuple('Point','x y')>>>p = Point(1, y=2); pPoint(x=1, y=2)>>>p[0]1>>>p.x1>>>getattr(p,'y')2

#Range

Immutable and hashable sequence of integers.

<range> = range(stop)# range(to_exclusive)<range> = range(start, stop)# range(from_inclusive, to_exclusive)<range> = range(start, stop, ±step)# range(from_inclusive, to_exclusive, ±step_size)

#Enumerate

for i, elin enumerate(<coll>, start=0):# Returns next element and its index on each pass.    ...

#Iterator

<iter> = iter(<collection>)# `iter(<iter>)` returns unmodified iterator.<iter> = iter(<function>, to_exclusive)# A sequence of return values until 'to_exclusive'.<el>   = next(<iter> [, default])# Raises StopIteration or returns 'default' on end.<list> = list(<iter>)# Returns a list of iterator's remaining elements.

Itertools

import itertoolsas it

#Generator

• Any function that contains a yield statement returns a generator.
• Generators and iterators are interchangeable.

defcount(start, step):whileTrue:yield start        start += step

#Type

• Everything is an object.
• Every object has a type.
• Type and class are synonymous.

<type> = type(<el>)# Or: <el>.__class__<bool> = isinstance(<el>, <type>)# Or: issubclass(type(<el>), <type>)

Some types do not have built-in names, so they must be imported:

from typesimport FunctionType, MethodType, LambdaType, GeneratorType, ModuleType

Abstract Base Classes

Each abstract base class specifies a set of virtual subclasses. These classes are then recognized by isinstance() and issubclass() as subclasses of the ABC, although they are really not. ABC can also manually decide whether or not a specific class is its virtual subclass, usually based on which methods the class has implemented. For instance, Iterable ABC looks for method iter(), while Collection ABC looks for iter(), contains() and len().

>>>from collections.abcimport Iterable, Collection, Sequence>>>isinstance([1,2,3], Iterable)True

#String

Immutable sequence of characters.

<str>  = <str>.strip()# Strips all whitespace characters from both ends.<str>  = <str>.strip('<chars>')# Strips passed characters. Also lstrip/rstrip().

Property Methods

<bool> = <str>.isdecimal()# Checks for [0-9]. Also [०-९] and [٠-٩].<bool> = <str>.isdigit()# Checks for [²³¹…] and isdecimal().<bool> = <str>.isnumeric()# Checks for [¼½¾…], [零〇一…] and isdigit().<bool> = <str>.isalnum()# Checks for [a-zA-Z…] and isnumeric().<bool> = <str>.isprintable()# Checks for [ !#$%…] and isalnum().<bool> = <str>.isspace()# Checks for [ \t\n\r\f\v\x1c-\x1f\x85\xa0…].

#Regex

Functions for regular expression matching.

import re<str>   = re.sub(r'<regex>', new, text, count=0)# Substitutes all occurrences with 'new'.<list>  = re.findall(r'<regex>', text)# Returns all occurrences as strings.<list>  = re.split(r'<regex>', text, maxsplit=0)# Add brackets around regex to keep matches.<Match> = re.search(r'<regex>', text)# First occurrence of the pattern or None.<Match> = re.match(r'<regex>', text)# Searches only at the beginning of the text.<iter>  = re.finditer(r'<regex>', text)# Returns all occurrences as Match objects.

Match Object

<str>   = <Match>.group()# Returns the whole match. Also group(0).<str>   = <Match>.group(1)# Returns part inside the first brackets.<tuple> = <Match>.groups()# Returns all bracketed parts.<int>   = <Match>.start()# Returns start index of the match.<int>   = <Match>.end()# Returns exclusive end index of the match.

Special Sequences

'\d' =='[0-9]'# Also [०-९…]. Matches a decimal character.'\w' =='[a-zA-Z0-9_]'# Also [ª²³…]. Matches an alphanumeric or _.'\s' =='[ \t\n\r\f\v]'# Also [\x1c-\x1f…]. Matches a whitespace.

#Format

<str> =f'{<el_1>},{<el_2>}'# Curly brackets can also contain expressions.<str> ='{}, {}'.format(<el_1>, <el_2>)# Or: '{0}, {a}'.format(<el_1>, a=<el_2>)<str> ='%s, %s' % (<el_1>, <el_2>)# Redundant and inferior C-style formatting.

Example

>>>Person = collections.namedtuple('Person','name height')>>>person = Person('Jean-Luc',187)>>>f'{person.name} is{person.height /100} meters tall.''Jean-Luc is 1.87 meters tall.'

General Options

{<el>:<10}# '<el>      '{<el>:^10}# '   <el>   '{<el>:>10}# '      <el>'{<el>:.<10}# '<el>......'{<el>:0}# '<el>'

Strings

{'abcde':10}# 'abcde     '{'abcde':10.3}# 'abc       '{'abcde':.3}# 'abc'{'abcde'!r:10}# "'abcde'   "

Numbers

{123456:10}# '    123456'{123456:10,}# '   123,456'{123456:10_}# '   123_456'{123456:+10}# '   +123456'{123456:=+10}# '+   123456'{123456: }# ' 123456'{-123456: }# '-123456'

Floats

{1.23456:10.3}# '      1.23'{1.23456:10.3f}# '     1.235'{1.23456:10.3e}# ' 1.235e+00'{1.23456:10.3%}# '  123.456%'

Comparison of presentation types:

┏━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓┃              │    {<float>}   │   {<float>:f}  │   {<float>:e}  │   {<float>:%}  ┃┠──────────────┼────────────────┼────────────────┼────────────────┼────────────────┨┃  0.000056789 │   '5.6789e-05' │    '0.000057'  │ '5.678900e-05' │    '0.005679%' ┃┃  0.00056789  │   '0.00056789' │    '0.000568'  │ '5.678900e-04' │    '0.056789%' ┃┃  0.0056789   │   '0.0056789'  │    '0.005679'  │ '5.678900e-03' │    '0.567890%' ┃┃  0.056789    │   '0.056789'   │    '0.056789'  │ '5.678900e-02' │    '5.678900%' ┃┃  0.56789     │   '0.56789'    │    '0.567890'  │ '5.678900e-01' │   '56.789000%' ┃┃  5.6789      │   '5.6789'     │    '5.678900'  │ '5.678900e+00' │  '567.890000%' ┃┃ 56.789       │  '56.789'      │   '56.789000'  │ '5.678900e+01' │ '5678.900000%' ┃┗━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛┏━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓┃              │  {<float>:.2}  │  {<float>:.2f} │  {<float>:.2e} │  {<float>:.2%} ┃┠──────────────┼────────────────┼────────────────┼────────────────┼────────────────┨┃  0.000056789 │    '5.7e-05'   │      '0.00'    │   '5.68e-05'   │      '0.01%'   ┃┃  0.00056789  │    '0.00057'   │      '0.00'    │   '5.68e-04'   │      '0.06%'   ┃┃  0.0056789   │    '0.0057'    │      '0.01'    │   '5.68e-03'   │      '0.57%'   ┃┃  0.056789    │    '0.057'     │      '0.06'    │   '5.68e-02'   │      '5.68%'   ┃┃  0.56789     │    '0.57'      │      '0.57'    │   '5.68e-01'   │     '56.79%'   ┃┃  5.6789      │    '5.7'       │      '5.68'    │   '5.68e+00'   │    '567.89%'   ┃┃ 56.789       │    '5.7e+01'   │     '56.79'    │   '5.68e+01'   │   '5678.90%'   ┃┗━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛

Ints

{90:c}# 'Z'. Unicode character with value 90.{90:b}# '1011010'. Number 90 in binary.{90:X}# '5A'. Number 90 in uppercase hexadecimal.

#Numbers

<int>      = int(<float/str/bool>)# Or: math.trunc(<float>)<float>    = float(<int/str/bool>)# Or: <int/float>e±<int><complex>  = complex(real=0, imag=0)# Or: <int/float> ± <int/float>j<Fraction> = fractions.Fraction(0,1)# Or: Fraction(numerator=0, denominator=1)<Decimal>  = decimal.Decimal(<str/int>)# Or: Decimal((sign, digits, exponent))

Basic Functions

<num> = pow(<num>, <num>)# Or: <number> ** <number><num> = abs(<num>)# <float> = abs(<complex>)<num> = round(<num> [, ±ndigits])# `round(126, -1) == 130`

Math

from mathimport e, pi, inf, nan, isinf, isnan# `<el> == nan` is always False.from mathimport sin, cos, tan, asin, acos, atan# Also: degrees, radians.from mathimport log, log10, log2# Log can accept base as second arg.

Statistics

from statisticsimport mean, median, variance# Also: stdev, quantiles, groupby.

Random

from randomimport random, randint, uniform# Also: gauss, choice, shuffle, seed.

Hexadecimal Numbers

<int> = ±0x<hex># Or: ±0b<bin><int> = int('±<hex>',16)# Or: int('±<bin>', 2)<int> = int('±0x<hex>',0)# Or: int('±0b<bin>', 0)<str> = hex(<int>)# Returns '[-]0x<hex>'. Also bin().

Bitwise Operators

<int> = <int> & <int># And (0b1100 & 0b1010 == 0b1000).<int> = <int> | <int># Or  (0b1100 | 0b1010 == 0b1110).<int> = <int> ^ <int># Xor (0b1100 ^ 0b1010 == 0b0110).<int> = <int> << n_bits# Left shift. Use >> for right.<int> = ~<int># Not. Also -<int> - 1.

#Combinatorics

import itertoolsas it

#Datetime

Provides 'date', 'time', 'datetime' and 'timedelta' classes. All are immutable and hashable.

# $ pip3 install python-dateutilfrom datetimeimport date, time, datetime, timedelta, timezoneimport zoneinfo, dateutil.tz

Now

<D/DTn> = D/DT.today()# Current local date or naive DT. Also DT.now().<DTa>   = DT.now(<tzinfo>)# Aware DT from current time in passed timezone.

Timezone

<tzinfo> = timezone.utc# London without daylight saving time (DST).<tzinfo> = timezone(<timedelta>)# Timezone with fixed offset from UTC.<tzinfo> = dateutil.tz.tzlocal()# Local timezone with dynamic offset from UTC.<tzinfo> = zoneinfo.ZoneInfo('<iana_key>')# 'Continent/City_Name' zone with dynamic offset.<DTa>    = <DT>.astimezone([<tzinfo>])# Converts DT to the passed or local fixed zone.<Ta/DTa> = <T/DT>.replace(tzinfo=<tzinfo>)# Changes object's timezone without conversion.

Encode

<D/T/DT> = D/T/DT.fromisoformat(<str>)# Object from ISO string. Raises ValueError.<DT>     = DT.strptime(<str>,'<format>')# Datetime from str, according to format.<D/DTn>  = D/DT.fromordinal(<int>)# D/DT from days since the Gregorian NYE 1.<DTn>    = DT.fromtimestamp(<float>)# Local naive DT from seconds since the Epoch.<DTa>    = DT.fromtimestamp(<float>, <tz>)# Aware datetime from seconds since the Epoch.

Decode

<str>    = <D/T/DT>.isoformat(sep='T')# Also `timespec='auto/hours/minutes/seconds/…'`.<str>    = <D/T/DT>.strftime('<format>')# Custom string representation of the object.<int>    = <D/DT>.toordinal()# Days since Gregorian NYE 1, ignoring time and tz.<float>  = <DTn>.timestamp()# Seconds since the Epoch, from local naive DT.<float>  = <DTa>.timestamp()# Seconds since the Epoch, from aware datetime.

Format

>>>dt = datetime.strptime('2025-08-14 23:39:00.00 +0200','%Y-%m-%d %H:%M:%S.%f %z')>>>dt.strftime("%dth of %B '%y (%a), %I:%M %p %Z")"14th of August '25 (Thu), 11:39 PM UTC+02:00"

Arithmetics

<bool>   = <D/T/DTn> > <D/T/DTn># Ignores time jumps (fold attribute). Also ==.<bool>   = <DTa>     > <DTa># Ignores time jumps if they share tzinfo object.<TD>     = <D/DTn>   - <D/DTn># Ignores jumps. Convert to UTC for actual delta.<TD>     = <DTa>     - <DTa># Ignores jumps if they share tzinfo object.<D/DT>   = <D/DT>    ± <TD># Returned datetime can fall into missing hour.<TD>     = <TD>      * <float># Also `<TD> = abs(<TD>)`, `<TD> = <TD> ± <TD>`.<float>  = <TD>      / <TD># Also `(<int>, <TD>) = divmod(<TD>, <TD>)`.

#Function

Independent block of code that returns a value when called.

def <func_name>(<nondefault_args>): ...# E.g. `def func(x, y): ...`.def <func_name>(<default_args>): ...# E.g. `def func(x=0, y=0): ...`.def <func_name>(<nondefault_args>, <default_args>): ...# E.g. `def func(x, y=0): ...`.

Function Call

<obj> = <function>(<positional_args>)# E.g. `func(0, 0)`.<obj> = <function>(<keyword_args>)# E.g. `func(x=0, y=0)`.<obj> = <function>(<positional_args>, <keyword_args>)# E.g. `func(0, y=0)`.

#Splat Operator

Splat expands a collection into positional arguments, while splatty-splat expands a dictionary into keyword arguments.

args, kwargs = (1,2), {'z':3}func(*args, **kwargs)

Is the same as:

func(1,2, z=3)

Inside Function Definition

Splat combines zero or more positional arguments into a tuple, while splatty-splat combines zero or more keyword arguments into a dictionary.

>>>defadd(*a):...return sum(a)...>>>add(1,2,3)6

Allowed compositions of arguments inside function definition and the ways they can be called:

┏━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━┓┃                    │ f(1,2,3) │ f(1,2, z=3) │ f(1, y=2, z=3) │ f(x=1, y=2, z=3) ┃┠────────────────────┼────────────┼──────────────┼────────────────┼──────────────────┨┃f(x, *args, **kw): │      ✓     │      ✓       │        ✓       │        ✓         ┃┃f(*args, z, **kw): │            │      ✓       │        ✓       │        ✓         ┃┃f(x, **kw):        │            │              │        ✓       │        ✓         ┃┃f(*, x, **kw):     │            │              │                │        ✓         ┃┗━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━┛

Other Uses

<list>  = [*<collection> [, ...]]# Or: list(<coll>) [+ ...]<tuple> = (*<collection>, [...])# Or: tuple(<coll>) [+ ...]<set>   = {*<collection> [, ...]}# Or: set(<coll>) [| ...]<dict>  = {**<dict> [, ...]}# Or: <dict> | ...

#Inline

<func> =lambda: <return_value># A single statement function.<func> =lambda <arg_1>, <arg_2>: <return_value># Also allows default arguments.

Comprehensions

<list> = [i+1for iin range(10)]# Or: [1, 2, ..., 10]<iter> = (ifor iin range(10)if i >5)# Or: iter([6, 7, 8, 9])<set>  = {i+5for iin range(10)}# Or: {5, 6, ..., 14}<dict> = {i: i*2for iin range(10)}# Or: {0: 0, 1: 2, ..., 9: 18}

Map, Filter, Reduce

from functoolsimport reduce

Any, All

<bool> = any(<collection>)# Is `bool(<el>)` True for any el?<bool> = all(<collection>)# True for all? Also True if empty.

Conditional Expression

<obj> = <exp>if <condition>else <exp># Only one expression is evaluated.

And, Or

<obj> = <exp>and <exp> [and ...]# Returns first false or last operand.<obj> = <exp>or <exp> [or ...]# Returns first true or last operand.

Walrus Operator

>>>[ifor ain'0123'if (i := int(a)) >0]# Assigns to variable mid-sentence.[1,2,3]

Named Tuple, Enum, Dataclass

from collectionsimport namedtuplePoint = namedtuple('Point','x y')# Creates tuple's subclass.point = Point(0,0)# Returns its instance.from enumimport EnumDirection = Enum('Direction','N E S W')# Creates Enum's subclass.direction = Direction.N# Returns its member.from dataclassesimport make_dataclassPlayer = make_dataclass('Player', ['loc','dir'])# Creates a class.player = Player(point, direction)# Returns its instance.

#Imports

Mechanism that makes code in one file available to another file.

import <module># Imports a built-in or '<module>.py'.import <package># Imports a built-in or '<package>/__init__.py'.import <package>.<module># Imports a built-in or '<package>/<module>.py'.

#Closure

We have/get a closure in Python when a nested function references a value of its enclosing function and then the enclosing function returns its nested function.

defget_multiplier(a):defout(b):return a * breturn out

Partial

from functoolsimport partial<function> = partial(<function> [, <arg_1> [, ...]])

Non-Local

If variable is being assigned to anywhere in the scope, it is regarded as a local variable, unless it is declared as a 'global' or a 'nonlocal'.

defget_counter():    i =0defout():nonlocal i        i +=1return ireturn out

#Decorator

• A decorator takes a function, adds some functionality and returns it.
• It can be anycallable, but is usually implemented as a function that returns aclosure.

@decorator_namedeffunction_that_gets_passed_to_decorator():    ...

Debugger Example

Decorator that prints function's name every time the function is called.

from functoolsimport wrapsdefdebug(func):    @wraps(func)defout(*args, **kwargs):        print(func.__name__)return func(*args, **kwargs)return out@debugdefadd(x, y):return x + y

Cache

Decorator that caches function's return values. All function's arguments must be hashable.

from functoolsimport cache@cachedeffib(n):return nif n <2else fib(n-2) + fib(n-1)

Parametrized Decorator

A decorator that accepts arguments and returns a normal decorator that accepts a function.

from functoolsimport wrapsdefdebug(print_result=False):defdecorator(func):        @wraps(func)defout(*args, **kwargs):            result = func(*args, **kwargs)            print(func.__name__, resultif print_resultelse'')return resultreturn outreturn decorator@debug(print_result=True)defadd(x, y):return x + y

#Class

A template for creating user-defined objects.

classMyClass:def__init__(self, a):        self.a = adef__str__(self):return str(self.a)def__repr__(self):        class_name = self.__class__.__name__returnf'{class_name}({self.a!r})'    @classmethoddefget_class_name(cls):return cls.__name__

Expressions that call the str() method:

print(<obj>)f'{<obj>}'logging.warning(<obj>)csv.writer(<file>).writerow([<obj>])raise Exception(<obj>)

Expressions that call the repr() method:

print/str/repr([<obj>])print/str/repr({<obj>: <obj>})f'{<obj>!r}'Z = make_dataclass('Z', ['a']); print/str/repr(Z(<obj>))>>><obj>

Inheritance

classPerson:def__init__(self, name):        self.name = nameclassEmployee(Person):def__init__(self, name, staff_num):        super().__init__(name)        self.staff_num = staff_num

Multiple inheritance:

classA:passclassB:passclassC(A, B):pass

Type Annotations

• They add type hints to variables, arguments and functions ('def f() -> <type>:').
• Hints are used by type checkers likemypy, data validation libraries such asPydantic and lately also byCython compiler. However, they are not enforced by CPython interpreter.

from collectionsimport abc<name>: <type> [| ...] [= <obj>]<name>: list/set/abc.Iterable/abc.Sequence[<type>] [= <obj>]<name>: dict/tuple[<type>, ...] [= <obj>]

Dataclass

Decorator that uses class variables to generate init(), repr() and eq() special methods.

from dataclassesimport dataclass, field, make_dataclass@dataclass(order=False, frozen=False)class <class_name>:    <attr_name>: <type>    <attr_name>: <type> = <default_value>    <attr_name>: list/dict/set = field(default_factory=list/dict/set)

Property

Pythonic way of implementing getters and setters.

classPerson:    @propertydefname(self):return' '.join(self._name)    @name.setterdefname(self, value):        self._name = value.split()

Slots

Mechanism that restricts objects to attributes listed in 'slots', reduces their memory footprint.

classMyClassWithSlots:    __slots__ = ['a']def__init__(self):        self.a =1

Copy

from copyimport copy, deepcopy<object> = copy/deepcopy(<object>)

#Duck Types

A duck type is an implicit type that prescribes a set of special methods. Any object that has those methods defined is considered a member of that duck type.

classMyComparable:def__init__(self, a):        self.a = adef__eq__(self, other):if isinstance(other, type(self)):return self.a == other.areturnNotImplemented

Hashable

• Hashable object needs both hash() and eq() methods and its hash value should never change.
• Hashable objects that compare equal must have the same hash value, meaning default hash() that returns'id(self)' will not do.
• That is why Python automatically makes classes unhashable if you only implement eq().

classMyHashable:def__init__(self, a):        self._a = a    @propertydefa(self):return self._adef__eq__(self, other):if isinstance(other, type(self)):return self.a == other.areturnNotImplementeddef__hash__(self):return hash(self.a)

Sortable

• With 'total_ordering' decorator, you only need to provide eq() and one of lt(), gt(), le() or ge() special methods and the rest will be automatically generated.
• Functions sorted() and min() only require lt() method, while max() only requires gt(). However, it is best to define them all so that confusion doesn't arise in other contexts.
• When two lists, strings or dataclasses are compared, their values get compared in order until a pair of unequal values is found. The comparison of this two values is then returned. The shorter sequence is considered smaller in case of all values being equal.
• To sort collection of strings in proper alphabetical order pass'key=locale.strxfrm' to sorted() after running'locale.setlocale(locale.LC_COLLATE, "en_US.UTF-8")'.

from functoolsimport total_ordering@total_orderingclassMySortable:def__init__(self, a):        self.a = adef__eq__(self, other):if isinstance(other, type(self)):return self.a == other.areturnNotImplementeddef__lt__(self, other):if isinstance(other, type(self)):return self.a < other.areturnNotImplemented

Iterator

• Any object that has methods next() and iter() is an iterator.
• Next() should return next item or raise StopIteration exception.
• Iter() should return 'self', i.e. unmodified object on which it was called.

classCounter:def__init__(self):        self.i =0def__next__(self):        self.i +=1return self.idef__iter__(self):return self

Python has many different iterator objects:

• Sequence iterators returned by theiter() function, such as list_iterator and set_iterator.
• Objects returned by theitertools module, such as count, repeat and cycle.
• Generators returned by thegenerator functions andgenerator expressions.
• File objects returned by theopen() function, etc.

classCounter:def__init__(self):        self.i =0def__call__(self):        self.i +=1return self.i

Context Manager

• With statements only work on objects that have enter() and exit() special methods.
• Enter() should lock the resources and optionally return an object.
• Exit() should release the resources.
• Any exception that happens inside the with block is passed to the exit() method.
• The exit() method can suppress the exception by returning a true value.

classMyOpen:def__init__(self, filename):        self.filename = filenamedef__enter__(self):        self.file = open(self.filename)return self.filedef__exit__(self, exc_type, exception, traceback):        self.file.close()

#Iterable Duck Types

classMyIterable:def__init__(self, a):        self.a = adef__iter__(self):return iter(self.a)def__contains__(self, el):return elin self.a

Collection

• Only required methods are iter() and len(). Len() should return the number of items.
• This cheatsheet actually means'<iterable>' when it uses'<collection>'.
• I chose not to use the name 'iterable' because it sounds scarier and more vague than 'collection'. The main drawback of this decision is that the reader could think a certain function doesn't accept iterators when it does, since iterators are the only built-in objects that are iterable but are not collections.

classMyCollection:def__init__(self, a):        self.a = adef__iter__(self):return iter(self.a)def__contains__(self, el):return elin self.adef__len__(self):return len(self.a)

Sequence

• Only required methods are getitem() and len().
• Getitem() should return an item at the passed index or raise IndexError.
• Iter() and contains() automatically work on any object that has getitem() defined.
• Reversed() automatically works on any object that has getitem() and len() defined. It returns reversed iterator of object's items.

classMySequence:def__init__(self, a):        self.a = adef__iter__(self):return iter(self.a)def__contains__(self, el):return elin self.adef__len__(self):return len(self.a)def__getitem__(self, i):return self.a[i]def__reversed__(self):return reversed(self.a)

Discrepancies between glossary definitions and abstract base classes:

• Python's glossary defines iterable as any object with special methods iter() and/or getitem() and sequence as any object with getitem() and len(). It doesn't define collection.
• Passing ABC Iterable to isinstance() or issubclass() only checks whether object/class has special method iter(), while ABC Collection checks for iter(), contains() and len().

ABC Sequence

• It's a richer interface than the basic sequence.
• Extending it generates iter(), contains(), reversed(), index() and count().
• Unlike'abc.Iterable' and'abc.Collection', it is not a duck type. That is why'issubclass(MySequence, abc.Sequence)' would return False even if MySequence had all the methods defined. It however recognizes list, tuple, range, str, bytes, bytearray, array, memoryview and deque, since they are registered as Sequence's virtual subclasses.

from collectionsimport abcclassMyAbcSequence(abc.Sequence):def__init__(self, a):        self.a = adef__len__(self):return len(self.a)def__getitem__(self, i):return self.a[i]

Table of required and automatically available special methods:

┏━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━━━┓┃            │  Iterable  │ Collection │  Sequence  │ abc.Sequence ┃┠────────────┼────────────┼────────────┼────────────┼──────────────┨┃ iter()     │     !      │     !      │     ✓      │      ✓       ┃┃ contains() │     ✓      │     ✓      │     ✓      │      ✓       ┃┃ len()      │            │     !      │     !      │      !       ┃┃ getitem()  │            │            │     !      │      !       ┃┃ reversed() │            │            │     ✓      │      ✓       ┃┃ index()    │            │            │            │      ✓       ┃┃ count()    │            │            │            │      ✓       ┃┗━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━━┛

#Enum

Class of named constants called members.

from enumimport Enum, auto

Inline

Cutlery = Enum('Cutlery','FORK KNIFE SPOON')Cutlery = Enum('Cutlery', ['FORK','KNIFE','SPOON'])Cutlery = Enum('Cutlery', {'FORK':1,'KNIFE':2,'SPOON':3})

User-defined functions cannot be values, so they must be wrapped:

from functoolsimport partialLogicOp = Enum('LogicOp', {'AND': partial(lambda l, r: land r),'OR':  partial(lambda l, r: lor r)})

#Exceptions

try:    <code>except <exception>:    <code>

Complex Example

try:    <code_1>except <exception_a>:    <code_2_a>except <exception_b>:    <code_2_b>else:    <code_2_c>finally:    <code_3>

Catching Exceptions

except <exception>: ...except <exception>as <name>: ...except (<exception>, [...]): ...except (<exception>, [...])as <name>: ...

Raising Exceptions

raise <exception>raise <exception>()raise <exception>(<obj> [, ...])

Re-raising caught exception:

except <exception> [as <name>]:    ...raise

Exception Object

arguments = <name>.argsexc_type  = <name>.__class__filename  = <name>.__traceback__.tb_frame.f_code.co_filenamefunc_name = <name>.__traceback__.tb_frame.f_code.co_nameline      = linecache.getline(filename, <name>.__traceback__.tb_lineno)trace_str =''.join(traceback.format_tb(<name>.__traceback__))error_msg =''.join(traceback.format_exception(type(<name>), <name>, <name>.__traceback__))

Built-in Exceptions

BaseException ├── SystemExit# Raised by the sys.exit() function. ├── KeyboardInterrupt# Raised when the user hits the interrupt key (ctrl-c). └── Exception# User-defined exceptions should be derived from this class.      ├── ArithmeticError# Base class for arithmetic errors such as ZeroDivisionError.      ├── AssertionError# Raised by `assert <exp>` if expression returns false value.      ├── AttributeError# Raised when object doesn't have requested attribute/method.      ├── EOFError# Raised by input() when it hits an end-of-file condition.      ├── LookupError# Base class for errors when a collection can't find an item.      │    ├── IndexError# Raised when a sequence index is out of range.      │    └── KeyError# Raised when a dictionary key or set element is missing.      ├── MemoryError# Out of memory. May be too late to start deleting variables.      ├── NameError# Raised when nonexistent name (variable/func/class) is used.      │    └── UnboundLocalError# Raised when local name is used before it's being defined.      ├── OSError# Errors such as FileExistsError/TimeoutError (see #Open).      │    └── ConnectionError# Errors such as BrokenPipeError/ConnectionAbortedError.      ├── RuntimeError# Raised by errors that don't fall into other categories.      │    ├── NotImplementedEr…# Can be raised by abstract methods or by unfinished code.      │    └── RecursionError# Raised when the maximum recursion depth is exceeded.      ├── StopIteration# Raised when an empty iterator is passed to next().      ├── TypeError# When an argument of the wrong type is passed to function.      └── ValueError# When argument has the right type but inappropriate value.

Collections and their exceptions:

┏━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━┓┃           │    List    │    Set     │    Dict    ┃┠───────────┼────────────┼────────────┼────────────┨┃ getitem() │ IndexError │            │  KeyError  ┃┃ pop()     │ IndexError │  KeyError  │  KeyError  ┃┃ remove()  │ ValueError │  KeyError  │            ┃┃ index()   │ ValueError │            │            ┃┗━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━┛

Useful built-in exceptions:

raise TypeError('Argument is of the wrong type!')raise ValueError('Argument has the right type but an inappropriate value!')raise RuntimeError('I am too lazy to define my own exception!')

User-defined Exceptions

classMyError(Exception):passclassMyInputError(MyError):pass

#Exit

Exits the interpreter by raising SystemExit exception.

import syssys.exit()# Exits with exit code 0 (success).sys.exit(<int>)# Exits with the passed exit code.sys.exit(<obj>)# Prints to stderr and exits with 1.

#Print

print(<el_1>, ..., sep=' ', end='\n', file=sys.stdout, flush=False)

Pretty Print

from pprintimport pprintpprint(<collection>, width=80, depth=None, compact=False, sort_dicts=True)

#Input

<str> = input()

#Command Line Arguments

import sysscripts_path = sys.argv[0]arguments    = sys.argv[1:]

Argument Parser

from argparseimport ArgumentParser, FileTypep = ArgumentParser(description=<str>)# Returns a parser.p.add_argument('-<short_name>','--<name>', action='store_true')# Flag (defaults to False).p.add_argument('-<short_name>','--<name>', type=<type>)# Option (defaults to None).p.add_argument('<name>', type=<type>, nargs=1)# Mandatory first argument.p.add_argument('<name>', type=<type>, nargs='+')# Mandatory remaining args.p.add_argument('<name>', type=<type>, nargs='?/*')# Optional argument/s.args  = p.parse_args()# Exits on parsing error.<obj> = args.<name># Returns `<type>(<arg>)`.

#Open

Opens a file and returns the corresponding file object.

<file> = open(<path>, mode='r', encoding=None, newline=None)

Modes

• 'r' - Read. Used by default.
• 'w' - Write. Deletes existing contents.
• 'x' - Write or fail if the file already exists.
• 'a' - Append. Creates new file if it doesn't exist.
• 'w+' - Read and write. Deletes existing contents.
• 'r+' - Read and write from the start.
• 'a+' - Read and write from the end.
• 'b' - Binary mode ('rb','wb','xb', …).

<file>.seek(0)# Moves to the start of the file.<file>.seek(offset)# Moves 'offset' chars/bytes from the start.<file>.seek(0,2)# Moves to the end of the file.<bin_file>.seek(±offset, origin)# Origin: 0 start, 1 current position, 2 end.

Read Text from File

defread_file(filename):with open(filename, encoding='utf-8')as file:return file.readlines()

Write Text to File

defwrite_to_file(filename, text):with open(filename,'w', encoding='utf-8')as file:        file.write(text)

#Paths

import os, globfrom pathlibimport Path

DirEntry

Unlike listdir(), scandir() returns DirEntry objects that cache isfile, isdir, and on Windows also stat information, thus significantly increasing the performance of code that requires it.

<iter> = os.scandir(path='.')# Returns DirEntry objects located at the path.<str>  = <DirEntry>.path# Returns the whole path as a string.<str>  = <DirEntry>.name# Returns final component as a string.<file> = open(<DirEntry>)# Opens the file and returns a file object.

Path Object

<Path> = Path(<path> [, ...])# Accepts strings, Paths, and DirEntry objects.<Path> = <path> / <path> [/ ...]# First or second path must be a Path object.<Path> = <Path>.resolve()# Returns absolute path with resolved symlinks.

#OS Commands

import os, shutil, subprocess

Shell Commands

<pipe> = os.popen('<commands>')# Executes commands in sh/cmd. Returns combined stdout.<str>  = <pipe>.read(size=-1)# Reads 'size' chars or until EOF. Also readline/s().<int>  = <pipe>.close()# Returns None if last command exited with returncode 0.

Sends '1 + 1' to the basic calculator and captures its output:

>>>subprocess.run('bc', input='1 + 1\n', capture_output=True, text=True)CompletedProcess(args='bc', returncode=0, stdout='2\n', stderr='')

Sends test.in to the basic calculator running in standard mode and saves its output to test.out:

>>>from shleximport split>>>os.popen('echo 1 + 1 > test.in')>>>subprocess.run(split('bc -s'), stdin=open('test.in'), stdout=open('test.out','w'))CompletedProcess(args=['bc','-s'], returncode=0)>>>open('test.out').read()'2\n'

#JSON

Text file format for storing collections of strings and numbers.

import json<str>  = json.dumps(<list/dict>)# Converts collection to JSON string.<coll> = json.loads(<str>)# Converts JSON string to collection.

Read Collection from JSON File

defread_json_file(filename):with open(filename, encoding='utf-8')as file:return json.load(file)

Write Collection to JSON File

defwrite_to_json_file(filename, collection):with open(filename,'w', encoding='utf-8')as file:        json.dump(collection, file, ensure_ascii=False, indent=2)

#Pickle

Binary file format for storing Python objects.

import pickle<bytes>  = pickle.dumps(<object>)# Converts object to bytes object.<object> = pickle.loads(<bytes>)# Converts bytes object to object.

Read Object from Pickle File

defread_pickle_file(filename):with open(filename,'rb')as file:return pickle.load(file)

Write Object to Pickle File

defwrite_to_pickle_file(filename, an_object):with open(filename,'wb')as file:        pickle.dump(an_object, file)

#CSV

Text file format for storing spreadsheets.

import csv

Read

<reader> = csv.reader(<file>)# Also: `dialect='excel', delimiter=','`.<list>   = next(<reader>)# Returns next row as a list of strings.<list>   = list(<reader>)# Returns a list of remaining rows.

Write

<writer> = csv.writer(<file>)# Also: `dialect='excel', delimiter=','`.<writer>.writerow(<collection>)# Encodes objects using `str(<el>)`.<writer>.writerows(<coll_of_coll>)# Appends multiple rows.

Parameters

• 'dialect' - Master parameter that sets the default values. String or a 'csv.Dialect' object.
• 'delimiter' - A one-character string used to separate fields.
• 'lineterminator' - How writer terminates rows. Reader looks for '\n', '\r' and '\r\n'.
• 'quotechar' - Character for quoting fields containing delimiters, quotechars, '\n' or '\r'.
• 'escapechar' - Character for escaping quotechars.
• 'doublequote' - Whether quotechars inside fields are/get doubled or escaped.
• 'quoting' - 0: As necessary, 1: All, 2: All but numbers which are read as floats, 3: None.
• 'skipinitialspace' - Is space character at the start of the field stripped by the reader.

┏━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━┓┃                  │     excel    │   excel-tab  │     unix     ┃┠──────────────────┼──────────────┼──────────────┼──────────────┨┃ delimiter        │       ','    │      '\t'    │       ','    ┃┃ lineterminator   │    '\r\n'    │    '\r\n'    │      '\n'    ┃┃ quotechar        │       '"'    │       '"'    │       '"'    ┃┃ escapechar       │      None    │      None    │      None    ┃┃ doublequote      │      True    │      True    │      True    ┃┃ quoting          │         0    │         0    │         1    ┃┃ skipinitialspace │     False    │     False    │     False    ┃┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┛

Read Rows from CSV File

defread_csv_file(filename, **csv_params):with open(filename, encoding='utf-8', newline='')as file:return list(csv.reader(file, **csv_params))

Write Rows to CSV File

defwrite_to_csv_file(filename, rows, mode='w', **csv_params):with open(filename, mode, encoding='utf-8', newline='')as file:        writer = csv.writer(file, **csv_params)        writer.writerows(rows)

#SQLite

A server-less database engine that stores each database into its own file.

import sqlite3<conn> = sqlite3.connect(<path>)# Opens existing or new file. Also ':memory:'.<conn>.close()# Closes connection. Discards uncommitted data.

Read

<cursor> = <conn>.execute('<query>')# Can raise a subclass of sqlite3.Error.<tuple>  = <cursor>.fetchone()# Returns next row. Also next(<cursor>).<list>   = <cursor>.fetchall()# Returns remaining rows. Also list(<cursor>).

Write

<conn>.execute('<query>')# Can raise a subclass of sqlite3.Error.<conn>.commit()# Saves all changes since the last commit.<conn>.rollback()# Discards all changes since the last commit.

Or:

with <conn>:# Exits the block with commit() or rollback(),    <conn>.execute('<query>')# depending on whether any exception occurred.

Placeholders

<conn>.execute('<query>', <list/tuple>)# Replaces '?'s in query with values.<conn>.execute('<query>', <dict/namedtuple>)# Replaces ':<key>'s with values.<conn>.executemany('<query>', <coll_of_coll>)# Runs execute() multiple times.

Example

Values are not actually saved in this example because'conn.commit()' is omitted!

>>>conn = sqlite3.connect('test.db')>>>conn.execute('CREATE TABLE person (person_id INTEGER PRIMARY KEY, name, height)')>>>conn.execute('INSERT INTO person VALUES (NULL, ?, ?)', ('Jean-Luc',187)).lastrowid1>>>conn.execute('SELECT * FROM person').fetchall()[(1,'Jean-Luc',187)]

SQLAlchemy

Library for interacting with various DB systems via SQL, method chaining, or ORM.

# $ pip3 install sqlalchemyfrom sqlalchemyimport create_engine, text<engine> = create_engine('<url>')# Url: 'dialect://user:password@host/dbname'.<conn>   = <engine>.connect()# Creates a connection. Also <conn>.close().<cursor> = <conn>.execute(text('<query>'), …)# `<dict>`. Replaces ':<key>'s with values.with <conn>.begin(): ...# Exits the block with commit or rollback.

#Bytes

A bytes object is an immutable sequence of single bytes. Mutable version is called bytearray.

<bytes> =b'<str>'# Only accepts ASCII characters and \x00-\xff.<int>   = <bytes>[index]# Returns an int in range from 0 to 255.<bytes> = <bytes>[<slice>]# Returns bytes even if it has only one element.<bytes> = <bytes>.join(<coll_of_bytes>)# Joins elements using bytes as a separator.

Encode

<bytes> = bytes(<coll_of_ints>)# Ints must be in range from 0 to 255.<bytes> = bytes(<str>,'utf-8')# Encodes the string. Also <str>.encode().<bytes> = bytes.fromhex('<hex>')# Hex pairs can be separated by whitespaces.<bytes> = <int>.to_bytes(n_bytes, …)# `byteorder='big/little', signed=False`.

Decode

<list>  = list(<bytes>)# Returns ints in range from 0 to 255.<str>   = str(<bytes>,'utf-8')# Returns a string. Also <bytes>.decode().<str>   = <bytes>.hex()# Returns hex pairs. Accepts `sep=<str>`.<int>   = int.from_bytes(<bytes>, …)# `byteorder='big/little', signed=False`.

Read Bytes from File

defread_bytes(filename):with open(filename,'rb')as file:return file.read()

Write Bytes to File

defwrite_bytes(filename, bytes_obj):with open(filename,'wb')as file:        file.write(bytes_obj)

#Struct

• Module that performs conversions between a sequence of numbers and a bytes object.
• System’s type sizes, byte order, and alignment rules are used by default.

from structimport pack, unpack<bytes> = pack('<format>', <el_1> [, ...])# Packs objects according to format string.<tuple> = unpack('<format>', <bytes>)# Use iter_unpack() to get iterator of tuples.

For standard type sizes and manual alignment (padding) start format string with:

• '=' - System's byte order (usually little-endian).
• '<' - Little-endian (i.e. least significant byte first).
• '>' - Big-endian (also'!').

#Array

List that can only hold numbers of a predefined type. Available types and their minimum sizes in bytes are listed above. Type sizes and byte order are always determined by the system, however bytes of each element can be reversed with byteswap() method.

from arrayimport array

#Memory View

A sequence object that points to the memory of another bytes-like object. Each element can reference a single or multiple consecutive bytes, depending on format. Order and number of elements can be changed with slicing.

<mview> = memoryview(<bytes/bytearray/array>)# Immutable if bytes is passed, else mutable.<obj>   = <mview>[index]# Returns int or float. Bytes if format is 'c'.<mview> = <mview>[<slice>]# Returns memoryview with rearranged elements.<mview> = <mview>.cast('<typecode>')# Only works between B/b/c and other types.<mview>.release()# Releases memory buffer of the base object.

#Deque

List with efficient appends and pops from either side.

from collectionsimport deque

#Operator

Module of functions that provide the functionality of operators. Functions are grouped by operator precedence, from least to most binding. Functions and operators in lines 1, 3 and 5 are also ordered by precedence within a group.

import operatoras op

#Match Statement

Executes the first block with matching pattern. Added in Python 3.10.

match <object/expression>:case <pattern> [if <condition>]:        <code>    ...

Patterns

<value_pattern> =1/'abc'/True/None/math.pi# Matches the literal or a dotted name.<class_pattern> = <type>()# Matches any object of that type (or ABC).<wildcard_patt> = _# Matches any object. Useful in last case.<capture_patt>  = <name># Matches any object and binds it to name.<as_pattern>    = <pattern>as <name># Binds match to name. Also <type>(<name>).<or_pattern>    = <pattern> | <pattern> [| ...]# Matches any of the patterns.<sequence_patt> = [<pattern>, ...]# Matches sequence with matching items.<mapping_patt>  = {<value_pattern>: <patt>, ...}# Matches dictionary with matching items.<class_pattern> = <type>(<attr_name>=<patt>, ...)# Matches object with matching attributes.

Example

>>>from pathlibimport Path>>>match Path('/home/gto/python-cheatsheet/README.md'):...case Path(...        parts=['/','home', user, *_]...    )as pif p.name.lower().startswith('readme')and p.is_file():...        print(f'{p.name} is a readme file that belongs to user{user}.')README.md is a readme file that belongs to user gto.

#Logging

import loggingas log

Setup

log.basicConfig(    filename=None,# Logs to stderr or appends to file.    format='%(levelname)s:%(name)s:%(message)s',# Add '%(asctime)s' for local datetime.    level=log.WARNING,# Drops messages with lower priority.    handlers=[log.StreamHandler(sys.stderr)]# Uses FileHandler if filename is set.)

Creates a logger that writes all messages to a file and sends them to the root's handler that prints warnings or higher:

>>>logger = log.getLogger('my_module')>>>handler = log.FileHandler('test.log', encoding='utf-8')>>>handler.setFormatter(log.Formatter('%(asctime)s %(levelname)s:%(name)s:%(message)s'))>>>logger.addHandler(handler)>>>logger.setLevel('DEBUG')>>>log.basicConfig()>>>log.root.handlers[0].setLevel('WARNING')>>>logger.critical('Running out of disk space.')CRITICAL:my_module:Running out of disk space.>>>print(open('test.log').read())2023-02-07 23:21:01,430 CRITICAL:my_module:Running out of disk space.

#Introspection

<list> = dir()# Local names of variables, functions, classes and modules.<dict> = vars()# Dict of local names and their objects. Also locals().<dict> = globals()# Dict of global names and their objects, e.g. __builtin__.

#Threading

CPython interpreter can only run a single thread at a time. Using multiple threads won't result in a faster execution, unless at least one of the threads contains an I/O operation.

from threadingimport Thread, Lock, RLock, Semaphore, Event, Barrierfrom concurrent.futuresimport ThreadPoolExecutor, as_completed

Thread

<Thread> = Thread(target=<function>)# Use `args=<collection>` to set the arguments.<Thread>.start()# Starts the thread. Also <Thread>.is_alive().<Thread>.join()# Waits for the thread to finish executing.

Lock

<lock> = Lock/RLock()# RLock can only be released by acquirer.<lock>.acquire()# Waits for the lock to be available.<lock>.release()# Makes the lock available again.

Or:

with <lock>:# Enters the block by calling acquire() and    ...# exits it with release(), even on error.

Semaphore, Event, Barrier

<Semaphore> = Semaphore(value=1)# Lock that can be acquired by 'value' threads.<Event>     = Event()# Method wait() blocks until set() is called.<Barrier>   = Barrier(n_times)# Wait() blocks until it's called n times.

Queue

<Queue> = queue.Queue(maxsize=0)# A thread-safe first-in-first-out queue.<Queue>.put(<el>)# Blocks until queue stops being full.<Queue>.put_nowait(<el>)# Raises queue.Full exception if full.<el> = <Queue>.get()# Blocks until queue stops being empty.<el> = <Queue>.get_nowait()# Raises queue.Empty exception if empty.

Thread Pool Executor

<Exec> = ThreadPoolExecutor(max_workers=None)# Or: `with ThreadPoolExecutor() as <name>: ...`<iter> = <Exec>.map(<func>, <args_1>, ...)# Multithreaded and non-lazy map(). Keeps order.<Futr> = <Exec>.submit(<func>, <arg_1>, ...)# Creates a thread and returns its Future obj.<Exec>.shutdown()# Blocks until all threads finish executing.

#Coroutines

• Coroutines have a lot in common with threads, but unlike threads, they only give up control when they call another coroutine and they don’t use as much memory.
• Coroutine definition starts with'async' and its call with'await'.
• Use'asyncio.run(<coroutine>)' to start the first/main coroutine.

import asyncioas aio

Runs a terminal game where you control an asterisk that must avoid numbers:

import asyncio, collections, curses, curses.textpad, enum, randomP = collections.namedtuple('P','x y')# PositionD = enum.Enum('D','n e s w')# DirectionW, H =15,7# Width, Heightdefmain(screen):    curses.curs_set(0)# Makes cursor invisible.    screen.nodelay(True)# Makes getch() non-blocking.    asyncio.run(main_coroutine(screen))# Starts running asyncio code.asyncdefmain_coroutine(screen):    moves = asyncio.Queue()    state = {'*': P(0,0)} | {id_: P(W//2, H//2)for id_in range(10)}    ai    = [random_controller(id_, moves)for id_in range(10)]    mvc   = [human_controller(screen, moves), model(moves, state), view(state, screen)]    tasks = [asyncio.create_task(coro)for coroin ai + mvc]await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)asyncdefrandom_controller(id_, moves):whileTrue:        d = random.choice(list(D))        moves.put_nowait((id_, d))await asyncio.sleep(random.triangular(0.01,0.65))asyncdefhuman_controller(screen, moves):whileTrue:        key_mappings = {258: D.s,259: D.n,260: D.w,261: D.e}if d := key_mappings.get(screen.getch()):            moves.put_nowait(('*', d))await asyncio.sleep(0.005)asyncdefmodel(moves, state):while state['*']notin (state[id_]for id_in range(10)):        id_, d =await moves.get()        deltas = {D.n: P(0,-1), D.e: P(1,0), D.s: P(0,1), D.w: P(-1,0)}        state[id_] = P((state[id_].x + deltas[d].x) % W, (state[id_].y + deltas[d].y) % H)asyncdefview(state, screen):    offset = P(curses.COLS//2 - W//2, curses.LINES//2 - H//2)whileTrue:        screen.erase()        curses.textpad.rectangle(screen, offset.y-1, offset.x-1, offset.y+H, offset.x+W)for id_, pin state.items():            screen.addstr(offset.y + (p.y - state['*'].y + H//2) % H,                          offset.x + (p.x - state['*'].x + W//2) % W, str(id_))        screen.refresh()await asyncio.sleep(0.005)if __name__ =='__main__':    curses.wrapper(main)

Libraries

# $ pip3 install tqdm>>>import tqdm, time>>>for elin tqdm.tqdm([1,2,3], desc='Processing'):...    time.sleep(1)Processing: 100%|████████████████████| 3/3 [00:03<00:00,  1.00s/it]

#Plot

# $ pip3 install matplotlibimport matplotlib.pyplotas pltplt.plot/bar/scatter(x_data, y_data [, label=<str>])# Also plt.plot(y_data).plt.legend()# Adds a legend.plt.title/xlabel/ylabel(<str>)# Adds a title or label.plt.show()# Also plt.savefig(<path>).plt.clf()# Clears the plot.

#Table

# $ pip3 install tabulateimport csv, tabulatewith open('test.csv', encoding='utf-8', newline='')as file:    rows = list(csv.reader(file))print(tabulate.tabulate(rows, headers='firstrow'))

#Console App

# $ pip3 install windows-cursesimport curses, osfrom cursesimport A_REVERSE, KEY_UP, KEY_DOWN, KEY_LEFT, KEY_RIGHT, KEY_ENTERdefmain(screen):    ch, first, selected, paths =0,0,0, os.listdir()while ch != ord('q'):        height, width = screen.getmaxyx()        screen.erase()for y, filenamein enumerate(paths[first : first+height]):            color = A_REVERSEif filename == paths[selected]else0            screen.addnstr(y,0, filename, width-1, color)        ch = screen.getch()        selected -= (ch == KEY_UP)and (selected >0)        selected += (ch == KEY_DOWN)and (selected < len(paths)-1)        first = min(first, selected)        first = max(first, selected - (height-1))if chin [KEY_LEFT, KEY_RIGHT, KEY_ENTER, ord('\n'), ord('\r')]:            new_dir ='..'if ch == KEY_LEFTelse paths[selected]if os.path.isdir(new_dir):                os.chdir(new_dir)                first, selected, paths =0,0, os.listdir()if __name__ =='__main__':    curses.wrapper(main)

#GUI App

# $ pip3 install PySimpleGUIimport PySimpleGUIas sgtext_box = sg.Input(default_text='100', enable_events=True, key='-QUANTITY-')dropdown = sg.InputCombo(['g','kg','t'],'kg', readonly=True, enable_events=True, k='-UNIT-')label    = sg.Text('100 kg is 220.462 lbs.', key='-OUTPUT-')button   = sg.Button('Close')window   = sg.Window('Weight Converter', [[text_box, dropdown], [label], [button]])whileTrue:    event, values = window.read()if eventin [sg.WIN_CLOSED,'Close']:breaktry:        quantity = float(values['-QUANTITY-'])except ValueError:continue    unit = values['-UNIT-']    factors = {'g':0.001,'kg':1,'t':1000}    lbs = quantity * factors[unit] /0.45359237    window['-OUTPUT-'].update(value=f'{quantity}{unit} is{lbs:g} lbs.')window.close()

#Scraping

# $ pip3 install requests beautifulsoup4import requests, bs4, osresponse   = requests.get('https://en.wikipedia.org/wiki/Python_(programming_language)')document   = bs4.BeautifulSoup(response.text,'html.parser')table      = document.find('table', class_='infobox vevent')python_url = table.find('th', text='Website').next_sibling.a['href']logo_url   = table.find('img')['src']filename   = os.path.basename(logo_url)with open(filename,'wb')as file:    file.write(requests.get(f'https:{logo_url}').content)print(f'{python_url}, file://{os.path.abspath(filename)}')

Selenium

Library for scraping websites with dynamic content.

# $ pip3 install seleniumfrom seleniumimport webdriver<WebDrv> = webdriver.Chrome/Firefox/Safari/Edge()# Opens a browser. Also <WebDrv>.quit().<WebDrv>.get('<url>')# Also <WebDrv>.implicitly_wait(seconds).<str>  = <WebDrv>.page_source# Returns HTML of fully rendered page.<El>   = <WebDrv/El>.find_element('css selector', …)# '<tag>#<id>.<class>[<attr>="<val>"]…'.<list> = <WebDrv/El>.find_elements('xpath', …)# '//<tag>[@<attr>="<val>"]…'. See XPath.<str>  = <El>.get_attribute(<str>)# Property if exists. Also <El>.text.<El>.click/clear()# Also <El>.send_keys(<str>).

XPath — also available in lxml, Scrapy, and browser's console via'$x("<xpath>")':

<xpath>     = //<element>[/or // <element>]# /<child>, //<descendant>, /../<sibling><xpath>     = //<element>/following::<element># Next element. Also preceding/parent/…<element>   = <tag><conditions><index># `<tag> = */a/…`, `<index> = [1/2/…]`.<condition> = [<sub_cond> [and/or <sub_cond>]]# For negation use `not(<sub_cond>)`.<sub_cond>  = @<attr>[="<val>"]# `text()=`, `.=` match (complete) text.<sub_cond>  = contains(@<attr>,"<val>")# Is <val> a substring of attr's value?<sub_cond>  = [//]<element># Has matching child? Descendant if //.

#Web App

Flask is a micro web framework/server. If you just want to open a html file in a web browser use'webbrowser.open(<path>)' instead.

# $ pip3 install flaskimport flaskas fl

Static Request

@app.route('/img/<path:filename>')defserve_file(filename):return fl.send_from_directory('DIRNAME', filename)

Dynamic Request

@app.route('/<sport>')defserve_html(sport):return fl.render_template_string('<h1>{{title}}</h1>', title=sport)

REST Request

@app.post('/<sport>/odds')defserve_json(sport):    team = fl.request.form['team']return {'team': team,'odds': [2.09,3.74,3.68]}

Starts the app in its own thread and queries its REST API:

# $ pip3 install requests>>>import threading, requests>>>threading.Thread(target=app.run, daemon=True).start()>>>url ='http://localhost:5000/football/odds'>>>response = requests.post(url, data={'team':'arsenal f.c.'})>>>response.json(){'team':'arsenal f.c.','odds': [2.09,3.74,3.68]}

#Profiling

from timeimport perf_counterstart_time = perf_counter()...duration_in_seconds = perf_counter() - start_time

Timing a Snippet

>>>from timeitimport timeit>>>timeit('list(range(10000))', number=1000, globals=globals(), setup='pass')0.19373

Profiling by Line

$ pip3 install line_profiler$ echo '@profiledef main():    a = list(range(10000))    b = set(range(10000))main()' > test.py$ kernprof -lv test.pyLine #      Hits         Time  Per Hit   % Time  Line Contents==============================================================     1                                           @profile     2                                           def main():     3         1        253.4    253.4     32.2      a = list(range(10000))     4         1        534.1    534.1     67.8      b = set(range(10000))

Call and Flame Graphs

$ apt/brew install graphviz && pip3 install gprof2dot snakeviz# Or download installer.$ tail --lines=+2 test.py > test.py# Removes first line.$ python3 -m cProfile -o test.prof test.py# Runs built-in profiler.$ gprof2dot --format=pstats test.prof | dot -T png -o test.png# Generates call graph.$ xdg-open/open test.png# Displays call graph.$ snakeviz test.prof# Displays flame graph.

Sampling and Memory Profilers

┏━━━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━┯━━━━━━┓┃ pip3 install │   Target   │          How to run           │ Lines │ Live ┃┠──────────────┼────────────┼───────────────────────────────┼───────┼──────┨┃ pyinstrument │    CPU     │ pyinstrument test.py          │   ×   │  ×   ┃┃ py-spy       │    CPU     │ py-spy top -- python3 test.py │   ×   │  ✓   ┃┃ scalene      │ CPU+Memory │ scalene test.py               │   ✓   │  ×   ┃┃ memray       │   Memory   │ memray run --live test.py     │   ✓   │  ✓   ┃┗━━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━┷━━━━━━┛

#NumPy

Array manipulation mini-language. It can run up to one hundred times faster than the equivalent Python code. An even faster alternative that runs on a GPU is called CuPy.

# $ pip3 install numpyimport numpyas np

Indexing

<el>       = <2d>[row_index, col_index]# Or: <3d>[<int>, <int>, <int>]<1d_view>  = <2d>[row_index]# Or: <3d>[<int>, <int>, <slice>]<1d_view>  = <2d>[:, col_index]# Or: <3d>[<int>, <slice>, <int>]<2d_view>  = <2d>[from:to_row_i, from:to_col_i]# Or: <3d>[<int>, <slice>, <slice>]

Broadcasting

A set of rules by which NumPy functions operate on arrays of different shapes.

left  = np.array([0.1,0.6,0.8])# `left.shape  == (3,)`right = np.array([[0.1], [0.6], [0.8]])# `right.shape == (3, 1)`

1. If array shapes differ in length, left-pad the shorter shape with ones:

left  = np.array([[0.1,0.6,0.8]])# `left.shape  == (1, 3)`right = np.array([[0.1], [0.6], [0.8]])# `right.shape == (3, 1)`

2. If any dimensions differ in size, expand the ones that have size 1 by duplicating their elements:

left  = np.array([[0.1,0.6,0.8],# `left.shape  == (3, 3)`                  [0.1,0.6,0.8],                  [0.1,0.6,0.8]])right = np.array([[0.1,0.1,0.1],# `right.shape == (3, 3)`                  [0.6,0.6,0.6],                  [0.8,0.8,0.8]])

Example

>>>print(points := np.array([0.1,0.6,0.8]))[0.10.60.8]>>>print(wrapped_points := points.reshape(3,1))[[0.1] [0.6] [0.8]]>>>print(deltas := points - wrapped_points)[[0.0.50.7] [-0.50.0.2] [-0.7-0.20. ]]>>>deltas[range(3), range(3)] = np.inf>>>print(distances := np.abs(deltas))[[inf0.50.7] [0.5  inf0.2] [0.70.2  inf]]>>>print(distances.argmin(axis=1))[121]

#Image

# $ pip3 install pillowfrom PILimport Image

Modes

• 'L' - Lightness (greyscale image). Each pixel is an int between 0 and 255.
• 'RGB' - Red, green, blue (true color image). Each pixel is a tuple of three ints.
• 'RGBA' - RGB with alpha. Low alpha (i.e. forth int) makes pixel more transparent.
• 'HSV' - Hue, saturation, value. Three ints representing color in HSV color space.

WIDTH, HEIGHT =100,100n_pixels = WIDTH * HEIGHThues = (255 * i/n_pixelsfor iin range(n_pixels))img = Image.new('HSV', (WIDTH, HEIGHT))img.putdata([(int(h),255,255)for hin hues])img.convert('RGB').save('test.png')

Adds noise to the PNG image and displays it:

from randomimport randintadd_noise =lambda value: max(0, min(255, value + randint(-20,20)))img = Image.open('test.png').convert('HSV')img.putdata([(add_noise(h), s, v)for h, s, vin img.getdata()])img.show()

Image Draw

from PILimport ImageDraw<Draw> = ImageDraw.Draw(<Image>)# Object for adding 2D graphics to the image.<Draw>.point((x, y))# Draws a point. Truncates floats into ints.<Draw>.line((x1, y1, x2, y2 [, ...]))# For anti-aliasing use <Image>.resize((w, h)).<Draw>.arc((x1, y1, x2, y2), deg1, deg2)# Draws in clockwise dir. Also pieslice().<Draw>.rectangle((x1, y1, x2, y2))# Also rounded_rectangle(), regular_polygon().<Draw>.polygon((x1, y1, x2, y2, ...))# Last point gets connected to the first one.<Draw>.ellipse((x1, y1, x2, y2))# To rotate use <Image>.rotate(anticlock_deg).<Draw>.text((x, y), <str>, font=<Font>)# `<Font> = ImageFont.truetype(<path>, size)`.

#Animation

# $ pip3 install imageiofrom PILimport Image, ImageDrawimport imageioWIDTH, HEIGHT, R =126,126,10frames = []for velocityin range(1,16):    y = sum(range(velocity))    frame = Image.new('L', (WIDTH, HEIGHT))    draw = ImageDraw.Draw(frame)    draw.ellipse((WIDTH/2-R, y, WIDTH/2+R, y+R*2), fill='white')    frames.append(frame)frames += reversed(frames[1:-1])imageio.mimsave('test.gif', frames, duration=0.03)

#Audio

import wave

Sample Values

┏━━━━━━━━━━━┯━━━━━━━━━━━┯━━━━━━┯━━━━━━━━━━━┓┃ sampwidth │    min    │ zero │    max    ┃┠───────────┼───────────┼──────┼───────────┨┃1     │0 │128 │255 ┃┃2     │-32768 │0 │32767 ┃┃3     │-8388608 │0 │8388607 ┃┗━━━━━━━━━━━┷━━━━━━━━━━━┷━━━━━━┷━━━━━━━━━━━┛

Read Float Samples from WAV File

defread_wav_file(filename):defget_int(bytes_obj):        an_int = int.from_bytes(bytes_obj,'little', signed=(p.sampwidth !=1))return an_int -128 * (p.sampwidth ==1)with wave.open(filename)as file:        p = file.getparams()        frames = file.readframes(-1)    bytes_samples = (frames[i : i + p.sampwidth]for iin range(0, len(frames), p.sampwidth))return [get_int(b) / pow(2, (p.sampwidth *8) -1)for bin bytes_samples], p

Write Float Samples to WAV File

defwrite_to_wav_file(filename, samples_f, p=None, nchannels=1, sampwidth=2, framerate=44100):defget_bytes(a_float):        a_float = max(-1, min(1 -2e-16, a_float))        a_float += (p.sampwidth ==1)        a_float *= pow(2, (p.sampwidth *8) -1)return int(a_float).to_bytes(p.sampwidth,'little', signed=(p.sampwidth !=1))if pisNone:        p = wave._wave_params(nchannels, sampwidth, framerate,0,'NONE','not compressed')with wave.open(filename,'wb')as file:        file.setparams(p)        file.writeframes(b''.join(get_bytes(f)for fin samples_f))

Examples

from mathimport pi, sinsamples_f = (sin(i *2 * pi *440 /44100)for iin range(100_000))write_to_wav_file('test.wav', samples_f)

Adds noise to the WAV file:

from randomimport uniformsamples_f, params = read_wav_file('test.wav')samples_f = (f + uniform(-0.05,0.05)for fin samples_f)write_to_wav_file('test.wav', samples_f, p=params)

Plays the WAV file:

# $ pip3 install simpleaudiofrom simpleaudioimport play_bufferwith wave.open('test.wav')as file:    frames, p = file.readframes(-1), file.getparams()    play_buffer(frames, p.nchannels, p.sampwidth, p.framerate).wait_done()

Text to Speech

# $ pip3 install pyttsx3import pyttsx3engine = pyttsx3.init()engine.say('Sally sells seashells by the seashore.')engine.runAndWait()

#Synthesizer

# $ pip3 install simpleaudioimport itertoolsas it, math, array, simpleaudiodefplay_notes(notes, bpm=132, f=44100):    get_pause   =lambda n_beats: it.repeat(0, int(n_beats *60/bpm * f))    sin_f       =lambda i, hz: math.sin(i *2 * math.pi * hz / f)    get_wave    =lambda hz, n_beats: (sin_f(i, hz)for iin range(int(n_beats *60/bpm * f)))    get_hz      =lambda note:440 *2 ** ((int(note[:2]) -69) /12)    get_nbeats  =lambda note:1/2if'♩'in noteelse1/4if'♪'in noteelse1    get_samples =lambda n: get_wave(get_hz(n), get_nbeats(n))if nelse get_pause(1/4)    samples_f   = it.chain.from_iterable(get_samples(n)for nin notes.split(','))    samples_i   = array.array('h', (int(fl *5000)for flin samples_f))    simpleaudio.play_buffer(samples_i,1,2, f).wait_done()play_notes('83♩,81♪,,83♪,,78♪,,74♪,,78♪,,71♪,,,,83♪,,81♪,,83♪,,78♪,,74♪,,78♪,,71♪,,,,''83♩,85♪,,86♪,,85♪,,86♪,,83♪,,85♩,83♪,,85♪,,81♪,,83♪,,81♪,,83♪,,79♪,,83♪,,,,')

#Pygame

# $ pip3 install pygameimport pygameas pgpg.init()screen = pg.display.set_mode((500,500))rect = pg.Rect(240,240,20,20)whilenot pg.event.get(pg.QUIT):    deltas = {pg.K_UP: (0,-20), pg.K_RIGHT: (20,0), pg.K_DOWN: (0,20), pg.K_LEFT: (-20,0)}for eventin pg.event.get(pg.KEYDOWN):        dx, dy = deltas.get(event.key, (0,0))        rect = rect.move((dx, dy))    screen.fill(pg.Color('black'))    pg.draw.rect(screen, pg.Color('white'), rect)    pg.display.flip()pg.quit()

Rect

Object for storing rectangular coordinates.

<Rect> = pg.Rect(x, y, width, height)# Creates Rect object. Truncates passed floats.<int>  = <Rect>.x/y/centerx/centery/…# Top, right, bottom, left. Allows assignments.<tup.> = <Rect>.topleft/center/…# Topright, bottomright, bottomleft. Same.<Rect> = <Rect>.move((delta_x, delta_y))# Use move_ip() to move in-place.

Surface

Object for representing images.

<Surf> = pg.display.set_mode((width, height))# Opens new window and returns its surface.<Surf> = pg.Surface((width, height))# New RGB surface. RGBA if `flags=pg.SRCALPHA`.<Surf> = pg.image.load(<path/file>)# Loads the image. Format depends on source.<Surf> = pg.surfarray.make_surface(<np_array>)# Also `<np_arr> = surfarray.pixels3d(<Surf>)`.<Surf> = <Surf>.subsurface(<Rect>)# Creates a new surface from the cutout.

Sound

<Sound> = pg.mixer.Sound(<path/file/bytes>)# WAV file or bytes/array of signed shorts.<Sound>.play/stop()# Also set_volume(<float>) and fadeout(msec).

Basic Mario Brothers Example

import collections, dataclasses, enum, io, itertoolsas it, pygameas pg, urllib.requestfrom randomimport randintP = collections.namedtuple('P','x y')# PositionD = enum.Enum('D','n e s w')# DirectionW, H, MAX_S =50,50, P(5,10)# Width, Height, Max speeddefmain():defget_screen():        pg.init()return pg.display.set_mode((W*16, H*16))defget_images():        url ='https://gto76.github.io/python-cheatsheet/web/mario_bros.png'        img = pg.image.load(io.BytesIO(urllib.request.urlopen(url).read()))return [img.subsurface(get_rect(x,0))for xin range(img.get_width() //16)]defget_mario():        Mario = dataclasses.make_dataclass('Mario','rect spd facing_left frame_cycle'.split())return Mario(get_rect(1,1), P(0,0),False, it.cycle(range(3)))defget_tiles():        border = [(x, y)for xin range(W)for yin range(H)if xin [0, W-1]or yin [0, H-1]]        platforms = [(randint(1, W-2), randint(2, H-2))for _in range(W*H //10)]return [get_rect(x, y)for x, yin border + platforms]defget_rect(x, y):return pg.Rect(x*16, y*16,16,16)    run(get_screen(), get_images(), get_mario(), get_tiles())defrun(screen, images, mario, tiles):    clock = pg.time.Clock()    pressed = set()whilenot pg.event.get(pg.QUIT):        clock.tick(28)        pressed |= {e.keyfor ein pg.event.get(pg.KEYDOWN)}        pressed -= {e.keyfor ein pg.event.get(pg.KEYUP)}        update_speed(mario, tiles, pressed)        update_position(mario, tiles)        draw(screen, images, mario, tiles)defupdate_speed(mario, tiles, pressed):    x, y = mario.spd    x +=2 * ((pg.K_RIGHTin pressed) - (pg.K_LEFTin pressed))    x += (x <0) - (x >0)    y +=1if D.snotin get_boundaries(mario.rect, tiles)else (pg.K_UPin pressed) *-10    mario.spd = P(x=max(-MAX_S.x, min(MAX_S.x, x)), y=max(-MAX_S.y, min(MAX_S.y, y)))defupdate_position(mario, tiles):    x, y = mario.rect.topleft    n_steps = max(abs(s)for sin mario.spd)for _in range(n_steps):        mario.spd = stop_on_collision(mario.spd, get_boundaries(mario.rect, tiles))        x, y = x + (mario.spd.x / n_steps), y + (mario.spd.y / n_steps)        mario.rect.topleft = x, ydefget_boundaries(rect, tiles):    deltas = {D.n: P(0,-1), D.e: P(1,0), D.s: P(0,1), D.w: P(-1,0)}return {dfor d, deltain deltas.items()if rect.move(delta).collidelist(tiles) !=-1}defstop_on_collision(spd, bounds):return P(x=0if (D.win boundsand spd.x <0)or (D.ein boundsand spd.x >0)else spd.x,             y=0if (D.nin boundsand spd.y <0)or (D.sin boundsand spd.y >0)else spd.y)defdraw(screen, images, mario, tiles):    screen.fill((85,168,255))    mario.facing_left = mario.spd.x <0if mario.spd.xelse mario.facing_left    is_airborne = D.snotin get_boundaries(mario.rect, tiles)    image_index =4if is_airborneelse next(mario.frame_cycle)if mario.spd.xelse6    screen.blit(images[image_index + (mario.facing_left *9)], mario.rect)for tin tiles:        is_border = t.xin [0, (W-1)*16]or t.yin [0, (H-1)*16]        screen.blit(images[18if is_borderelse19], t)    pg.display.flip()if __name__ =='__main__':    main()

#Pandas

Data analysis library. For examples seePlotly.

# $ pip3 install pandas matplotlibimport pandasas pd, matplotlib.pyplotas plt

Series

Ordered dictionary with a name.

>>>s = pd.Series([1,2], index=['x','y'], name='a'); sx1y2Name: a, dtype: int64

Series — Aggregate, Transform, Map:

<el> = <S>.sum/max/mean/std/idxmax/count()# Or: <S>.agg(lambda <S>: <el>)<S>  = <S>.rank/diff/cumsum/ffill/interpol…()# Or: <S>.agg/transform(lambda <S>: <S>)<S>  = <S>.isna/fillna/isin([<el/coll>])# Or: <S>.agg/transform/map(lambda <el>: <el>)

DataFrame

Table with labeled rows and columns.

>>>df = pd.DataFrame([[1,2], [3,4]], index=['a','b'], columns=['x','y']); df   x  ya12b34

DataFrame — Merge, Join, Concat:

>>>df_2 = pd.DataFrame([[4,5], [6,7]], index=['b','c'], columns=['y','z']); df_2   y  zb45c67

DataFrame — Aggregate, Transform, Map:

<S>  = <DF>.sum/max/mean/std/idxmax/count()# Or: <DF>.apply/agg(lambda <S>: <el>)<DF> = <DF>.rank/diff/cumsum/ffill/interpo…()# Or: <DF>.apply/agg/transform(lambda <S>: <S>)<DF> = <DF>.isna/fillna/isin([<el/coll>])# Or: <DF>.applymap(lambda <el>: <el>)

Multi-Index

<DF> = <DF>.loc[row_key_1]# Or: <DF>.xs(row_key_1)<DF> = <DF>.loc[:, (slice(None), col_key_2)]# Or: <DF>.xs(col_key_2, axis=1, level=1)<DF> = <DF>.set_index(col_keys)# Creates index from cols. Also `append=False`.<DF> = <DF>.pivot_table(index=col_key/s)# `columns=key/s, values=key/s, aggfunc='mean'`.<S>  = <DF>.stack/unstack(level=-1)# Combines col keys with row keys or vice versa.

File Formats

<S/DF> = pd.read_json/pickle(<path/url/file>)# Also io.StringIO(<str>), io.BytesIO(<bytes>).<DF>   = pd.read_csv/excel(<path/url/file>)# Also `header/index_col/dtype/usecols/…=<obj>`.<list> = pd.read_html(<path/url/file>)# Raises ImportError if webpage has zero tables.<S/DF> = pd.read_parquet/feather/hdf(<path…>)# Function read_hdf() accepts `key=<s/df_name>`.<DF>   = pd.read_sql('<table/query>', <conn>)# Pass SQLite3/Alchemy connection (see #SQLite).

GroupBy

Object that groups together rows of a dataframe based on the value of the passed column.

<GB> = <DF>.groupby(col_key/s)# Splits DF into groups based on passed column.<DF> = <GB>.apply/filter(<func>)# Filter drops a group if func returns False.<DF> = <GB>.get_group(<el>)# Selects a group by grouping column's value.<S>  = <GB>.size()# S of group sizes. Same keys as get_group().<GB> = <GB>[col_key]# Single column GB. All operations return S.

Divides rows into groups and sums their columns. Result has a named index that creates column'z' on reset_index():

>>>df = pd.DataFrame([[1,2,3], [4,5,6], [7,8,6]], list('abc'), list('xyz'))>>>gb = df.groupby('z'); gb.apply(print)   x  y  za123   x  y  zb456c786>>>gb.sum()    x   yz31261113

Rolling

Object for rolling window calculations.

<RS/RDF/RGB> = <S/DF/GB>.rolling(win_size)# Also: `min_periods=None, center=False`.<RS/RDF/RGB> = <RDF/RGB>[col_key/s]# Or: <RDF/RGB>.<col_key><S/DF>       = <R>.mean/sum/max()# Or: <R>.apply/agg(<agg_func/str>)

#Plotly

# $ pip3 install plotly kaleido pandasimport plotly.expressas px, pandasas pd

Displays a line chart of total coronavirus deaths per million grouped by continent:

covid = pd.read_csv('https://raw.githubusercontent.com/owid/covid-19-data/8dde8ca49b''6e648c17dd420b2726ca0779402651/public/data/owid-covid-data.csv',                    usecols=['iso_code','date','population','total_deaths'])continents = pd.read_csv('https://gto76.github.io/python-cheatsheet/web/continents.csv',                         usecols=['Three_Letter_Country_Code','Continent_Name'])df = pd.merge(covid, continents, left_on='iso_code', right_on='Three_Letter_Country_Code')df = df.groupby(['Continent_Name','date']).sum().reset_index()df['Total Deaths per Million'] = df.total_deaths *1e6 / df.populationdf = df[df.date >'2020-03-14']df = df.rename({'date':'Date','Continent_Name':'Continent'}, axis='columns')px.line(df, x='Date', y='Total Deaths per Million', color='Continent').show()

Displays a multi-axis line chart of total coronavirus cases and changes in prices of Bitcoin, Dow Jones and gold:

# $ pip3 install pandas lxml selenium plotlyimport pandasas pd, selenium.webdriver, io, plotly.graph_objectsas godefmain():    covid, (bitcoin, gold, dow) = get_covid_cases(), get_tickers()    df = wrangle_data(covid, bitcoin, gold, dow)    display_data(df)defget_covid_cases():    url ='https://covid.ourworldindata.org/data/owid-covid-data.csv'    df = pd.read_csv(url, parse_dates=['date'])    df = df[df.location =='World']    s = df.set_index('date').total_casesreturn s.rename('Total Cases')defget_tickers():with selenium.webdriver.Chrome()as driver:        symbols = {'Bitcoin':'BTC-USD','Gold':'GC=F','Dow Jones':'%5EDJI'}for name, symbolin symbols.items():yield get_ticker(driver, name, symbol)defget_ticker(driver, name, symbol):    url =f'https://finance.yahoo.com/quote/{symbol}/history/'    driver.get(url +'?period1=1579651200&period2=9999999999')if buttons := driver.find_elements('xpath','//button[@name="reject"]'):        buttons[0].click()    html = io.StringIO(driver.page_source)    dataframes = pd.read_html(html, parse_dates=['Date'])    s = dataframes[0].set_index('Date').Openreturn s.rename(name)defwrangle_data(covid, bitcoin, gold, dow):    df = pd.concat([bitcoin, gold, dow], axis=1)# Creates table by joining columns on dates.    df = df.sort_index().interpolate()# Sorts rows by date and interpolates NaN-s.    df = df.loc['2020-02-23':'2021-12-20']# Keeps rows between specified dates.    df = (df / df.iloc[0]) *100# Calculates percentages relative to day 1.    df = df.join(covid)# Adds column with covid cases.return df.sort_values(df.index[-1], axis=1)# Sorts columns by last day's value.defdisplay_data(df):    figure = go.Figure()for col_namein reversed(df.columns):        yaxis ='y1'if col_name =='Total Cases'else'y2'        trace = go.Scatter(x=df.index, y=df[col_name], yaxis=yaxis, name=col_name)        figure.add_trace(trace)    figure.update_layout(        width=944,        height=423,        yaxis1=dict(title='Total Cases', rangemode='tozero'),        yaxis2=dict(title='%', rangemode='tozero', overlaying='y', side='right'),        colorway=['#EF553B','#636EFA','#00CC96','#FFA152'],        legend=dict(x=1.08)    )    figure.show()if __name__ =='__main__':    main()

#Appendix

# $ pip3 install cythonimport pyximport; pyximport.install()# Module that runs imported Cython scripts.import <cython_script># Script must be saved with '.pyx' extension.<cython_script>.main()# Main() isn't automatically executed.

Definitions:

• All'cdef' definitions are optional, but they contribute to the speed-up.
• Also supports C pointers (via'*' and'&'), structs, unions and enums.

cdef <ctype/type> <var_name> [= <obj>]cdef <ctype>[n_elements] <var_name> [= <coll_of_nums>]cdef <ctype/type/void> <func_name>(<ctype/type> <arg_name>): ...

Virtual Environments

System for installing libraries directly into project's directory.

$ python3 -m venv NAME# Creates virtual environment in current directory.$ source NAME/bin/activate# Activates it. On Windows run `NAME\Scripts\activate`.$ pip3 install LIBRARY# Installs the library into active environment.$ python3 FILE# Runs the script in active environment. Also `./FILE`.$ deactivate# Deactivates the active virtual environment.

Basic Script Template

Run the script with'$ python3 FILE' or'$ chmod u+x FILE; ./FILE'. To automatically start the debugger when uncaught exception occurs run'$ python3 -m pdb -cc FILE'.

#!/usr/bin/env python3## Usage: .py#from sysimport argv, exitfrom collectionsimport defaultdict, namedtuplefrom dataclassesimport make_dataclassfrom enumimport Enumimport functoolsas ft, itertoolsas it, operatoras op, redefmain():pass#####  UTIL#defread_file(filename):with open(filename, encoding='utf-8')as file:return file.readlines()if __name__ =='__main__':    main()

#Index

• Ctrl+F / ⌘F is usually sufficient.
• Searching'#<title>' will limit the search to the titles.
• Click on the title's'#' to get a link to its section.