What’s New In Python 3.10

Editor:

Pablo Galindo Salgado

This article explains the new features in Python 3.10, compared to 3.9.Python 3.10 was released on October 4, 2021.For full details, see thechangelog.

Summary – Release highlights

New syntax features:

  • PEP 634, Structural Pattern Matching: Specification

  • PEP 635, Structural Pattern Matching: Motivation and Rationale

  • PEP 636, Structural Pattern Matching: Tutorial

  • bpo-12782, Parenthesized context managers are now officially allowed.

New features in the standard library:

  • PEP 618, Add Optional Length-Checking To zip.

Interpreter improvements:

  • PEP 626, Precise line numbers for debugging and other tools.

New typing features:

  • PEP 604, Allow writing union types as X | Y

  • PEP 612, Parameter Specification Variables

  • PEP 613, Explicit Type Aliases

  • PEP 647, User-Defined Type Guards

Important deprecations, removals or restrictions:

  • PEP 644, Require OpenSSL 1.1.1 or newer

  • PEP 632, Deprecate distutils module.

  • PEP 623, Deprecate and prepare for the removal of the wstr member in PyUnicodeObject.

  • PEP 624, Remove Py_UNICODE encoder APIs

  • PEP 597, Add optional EncodingWarning

New Features

Parenthesized context managers

Using enclosing parentheses for continuation across multiple linesin context managers is now supported. This allows formatting a longcollection of context managers in multiple lines in a similar wayas it was previously possible with import statements. For instance,all these examples are now valid:

with(CtxManager()asexample):...with(CtxManager1(),CtxManager2()):...with(CtxManager1()asexample,CtxManager2()):...with(CtxManager1(),CtxManager2()asexample):...with(CtxManager1()asexample1,CtxManager2()asexample2):...

it is also possible to use a trailing comma at the end of theenclosed group:

with(CtxManager1()asexample1,CtxManager2()asexample2,CtxManager3()asexample3,):...

This new syntax uses the non LL(1) capacities of the new parser.CheckPEP 617 for more details.

(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaouinbpo-12782 andbpo-40334.)

Better error messages

SyntaxErrors

When parsing code that contains unclosed parentheses or brackets the interpreternow includes the location of the unclosed bracket of parentheses instead of displayingSyntaxError: unexpected EOF while parsing or pointing to some incorrect location.For instance, consider the following code (notice the unclosed “{“):

expected={9:1,18:2,19:2,27:3,28:3,29:3,36:4,37:4,38:4,39:4,45:5,46:5,47:5,48:5,49:5,54:6,some_other_code=foo()

Previous versions of the interpreter reported confusing places as the location ofthe syntax error:

File"example.py",line3some_other_code=foo()^SyntaxError:invalidsyntax

but in Python 3.10 a more informative error is emitted:

File"example.py",line1expected={9:1,18:2,19:2,27:3,28:3,29:3,36:4,37:4,^SyntaxError:'{'wasneverclosed

In a similar way, errors involving unclosed string literals (single and triplequoted) now point to the start of the string instead of reporting EOF/EOL.

These improvements are inspired by previous work in the PyPy interpreter.

(Contributed by Pablo Galindo inbpo-42864 and Batuhan Taskaya inbpo-40176.)

SyntaxError exceptions raised by the interpreter will now highlight thefull error range of the expression that constitutes the syntax error itself,instead of just where the problem is detected. In this way, instead of displaying(before Python 3.10):

>>>foo(x,zforzinrange(10),t,w)  File"<stdin>", line1foo(x,zforzinrange(10),t,w)^SyntaxError:Generator expression must be parenthesized

now Python 3.10 will display the exception as:

>>>foo(x,zforzinrange(10),t,w)  File"<stdin>", line1foo(x,zforzinrange(10),t,w)^^^^^^^^^^^^^^^^^^^^SyntaxError:Generator expression must be parenthesized

This improvement was contributed by Pablo Galindo inbpo-43914.

A considerable amount of new specialized messages forSyntaxError exceptionshave been incorporated. Some of the most notable ones are as follows:

  • Missing: before blocks:

    >>>ifrocket.position>event_horizon  File"<stdin>", line1ifrocket.position>event_horizon^SyntaxError:expected ':'

    (Contributed by Pablo Galindo inbpo-42997.)

  • Unparenthesised tuples in comprehensions targets:

    >>>{x,yforx,yinzip('abcd','1234')}  File"<stdin>", line1{x,yforx,yinzip('abcd','1234')}^SyntaxError:did you forget parentheses around the comprehension target?

    (Contributed by Pablo Galindo inbpo-43017.)

  • Missing commas in collection literals and between expressions:

    >>>items={...x:1,...y:2...z:3,  File"<stdin>", line3y:2^SyntaxError:invalid syntax. Perhaps you forgot a comma?

    (Contributed by Pablo Galindo inbpo-43822.)

  • Multiple Exception types without parentheses:

    >>>try:...build_dyson_sphere()...exceptNotEnoughScienceError,NotEnoughResourcesError:  File"<stdin>", line3exceptNotEnoughScienceError,NotEnoughResourcesError:^SyntaxError:multiple exception types must be parenthesized

    (Contributed by Pablo Galindo inbpo-43149.)

  • Missing: and values in dictionary literals:

    >>>values={...x:1,...y:2,...z:...}  File"<stdin>", line4z:^SyntaxError:expression expected after dictionary key and ':'>>>values={x:1,y:2,zw:3}  File"<stdin>", line1values={x:1,y:2,zw:3}^SyntaxError:':' expected after dictionary key

    (Contributed by Pablo Galindo inbpo-43823.)

  • try blocks withoutexcept orfinally blocks:

    >>>try:...x=2...something=3  File"<stdin>", line3something=3^^^^^^^^^SyntaxError:expected 'except' or 'finally' block

    (Contributed by Pablo Galindo inbpo-44305.)

  • Usage of= instead of== in comparisons:

    >>>ifrocket.position=event_horizon:  File"<stdin>", line1ifrocket.position=event_horizon:^SyntaxError:cannot assign to attribute here. Maybe you meant '==' instead of '='?

    (Contributed by Pablo Galindo inbpo-43797.)

  • Usage of* in f-strings:

    >>>f"Black holes{*all_black_holes} and revelations"  File"<stdin>", line1(*all_black_holes)^SyntaxError:f-string: cannot use starred expression here

    (Contributed by Pablo Galindo inbpo-41064.)

IndentationErrors

ManyIndentationError exceptions now have more context regarding what kind of blockwas expecting an indentation, including the location of the statement:

>>>deffoo():...iflel:...x=2  File"<stdin>", line3x=2^IndentationError:expected an indented block after 'if' statement in line 2

AttributeErrors

When printingAttributeError,PyErr_Display() will offersuggestions of similar attribute names in the object that the exception wasraised from:

>>>collections.namedtoploTraceback (most recent call last):  File"<stdin>", line1, in<module>AttributeError:module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?

(Contributed by Pablo Galindo inbpo-38530.)

Προειδοποίηση

Notice this won’t work ifPyErr_Display() is not called to display the errorwhich can happen if some other custom error display function is used. This is a commonscenario in some REPLs like IPython.

NameErrors

When printingNameError raised by the interpreter,PyErr_Display()will offer suggestions of similar variable names in the function that the exceptionwas raised from:

>>>schwarzschild_black_hole=None>>>schwarschild_black_holeTraceback (most recent call last):  File"<stdin>", line1, in<module>NameError:name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?

(Contributed by Pablo Galindo inbpo-38530.)

Προειδοποίηση

Notice this won’t work ifPyErr_Display() is not called to display the error,which can happen if some other custom error display function is used. This is a commonscenario in some REPLs like IPython.

PEP 626: Precise line numbers for debugging and other tools

PEP 626 brings more precise and reliable line numbers for debugging, profiling and coverage tools.Tracing events, with the correct line number, are generated for all lines of code executed and only for lines of code that are executed.

Thef_lineno attribute of frame objects will always contain theexpected line number.

Theco_lnotab attribute ofcode objects is deprecated andwill be removed in 3.12.Code that needs to convert from offset to line number should use the newco_lines() method instead.

PEP 634: Structural Pattern Matching

Structural pattern matching has been added in the form of amatch statementandcase statements of patterns with associated actions. Patternsconsist of sequences, mappings, primitive data types as well as class instances.Pattern matching enables programs to extract information from complex data types,branch on the structure of data, and apply specific actions based on differentforms of data.

Syntax and operations

The generic syntax of pattern matching is:

matchsubject:case<pattern_1>:<action_1>case<pattern_2>:<action_2>case<pattern_3>:<action_3>case_:<action_wildcard>

A match statement takes an expression and compares its value to successivepatterns given as one or more case blocks. Specifically, pattern matchingoperates by:

  1. using data with type and shape (thesubject)

  2. evaluating thesubject in thematch statement

  3. comparing the subject with each pattern in acase statementfrom top to bottom until a match is confirmed.

  4. executing the action associated with the pattern of the confirmedmatch

  5. If an exact match is not confirmed, the last case, a wildcard_,if provided, will be used as the matching case. If an exact match isnot confirmed and a wildcard case does not exist, the entire matchblock is a no-op.

Declarative approach

Readers may be aware of pattern matching through the simple example of matchinga subject (data object) to a literal (pattern) with the switch statement foundin C, Java or JavaScript (and many other languages). Often the switch statementis used for comparison of an object/expression with case statements containingliterals.

More powerful examples of pattern matching can be found in languages such asScala and Elixir. With structural pattern matching, the approach is «declarative» andexplicitly states the conditions (the patterns) for data to match.

While an «imperative» series of instructions using nested «if» statementscould be used to accomplish something similar to structural pattern matching,it is less clear than the «declarative» approach. Instead the «declarative»approach states the conditions to meet for a match and is more readable throughits explicit patterns. While structural pattern matching can be used in itssimplest form comparing a variable to a literal in a case statement, itstrue value for Python lies in its handling of the subject’s type and shape.

Simple pattern: match to a literal

Let’s look at this example as pattern matching in its simplest form: a value,the subject, being matched to several literals, the patterns. In the examplebelow,status is the subject of the match statement. The patterns areeach of the case statements, where literals represent request status codes.The associated action to the case is executed after a match:

defhttp_error(status):matchstatus:case400:return"Bad request"case404:return"Not found"case418:return"I'm a teapot"case_:return"Something's wrong with the internet"

If the above function is passed astatus of 418, «I’m a teapot» is returned.If the above function is passed astatus of 500, the case statement with_ will match as a wildcard, and «Something’s wrong with the internet» isreturned.Note the last block: the variable name,_, acts as awildcard and insuresthe subject will always match. The use of_ is optional.

You can combine several literals in a single pattern using| («or»):

case401|403|404:return"Not allowed"
Behavior without the wildcard

If we modify the above example by removing the last case block, the examplebecomes:

defhttp_error(status):matchstatus:case400:return"Bad request"case404:return"Not found"case418:return"I'm a teapot"

Without the use of_ in a case statement, a match may not exist. If nomatch exists, the behavior is a no-op. For example, ifstatus of 500 ispassed, a no-op occurs.

Patterns with a literal and variable

Patterns can look like unpacking assignments, and a pattern may be used to bindvariables. In this example, a data point can be unpacked to its x-coordinateand y-coordinate:

# point is an (x, y) tuplematchpoint:case(0,0):print("Origin")case(0,y):print(f"Y={y}")case(x,0):print(f"X={x}")case(x,y):print(f"X={x}, Y={y}")case_:raiseValueError("Not a point")

The first pattern has two literals,(0,0), and may be thought of as anextension of the literal pattern shown above. The next two patterns combine aliteral and a variable, and the variablebinds a value from the subject(point). The fourth pattern captures two values, which makes itconceptually similar to the unpacking assignment(x,y)=point.

Patterns and classes

If you are using classes to structure your data, you can use as a patternthe class name followed by an argument list resembling a constructor. Thispattern has the ability to capture class attributes into variables:

classPoint:x:inty:intdeflocation(point):matchpoint:casePoint(x=0,y=0):print("Origin is the point's location.")casePoint(x=0,y=y):print(f"Y={y} and the point is on the y-axis.")casePoint(x=x,y=0):print(f"X={x} and the point is on the x-axis.")casePoint():print("The point is located somewhere else on the plane.")case_:print("Not a point")
Patterns with positional parameters

You can use positional parameters with some builtin classes that provide anordering for their attributes (e.g. dataclasses). You can also define a specificposition for attributes in patterns by setting the__match_args__ specialattribute in your classes. If it’s set to («x», «y»), the following patternsare all equivalent (and all bind they attribute to thevar variable):

Point(1,var)Point(1,y=var)Point(x=1,y=var)Point(y=var,x=1)

Nested patterns

Patterns can be arbitrarily nested. For example, if our data is a shortlist of points, it could be matched like this:

matchpoints:case[]:print("No points in the list.")case[Point(0,0)]:print("The origin is the only point in the list.")case[Point(x,y)]:print(f"A single point{x},{y} is in the list.")case[Point(0,y1),Point(0,y2)]:print(f"Two points on the Y axis at{y1},{y2} are in the list.")case_:print("Something else is found in the list.")

Complex patterns and the wildcard

To this point, the examples have used_ alone in the last case statement.A wildcard can be used in more complex patterns, such as('error',code,_).For example:

matchtest_variable:case('warning',code,40):print("A warning has been received.")case('error',code,_):print(f"An error{code} occurred.")

In the above case,test_variable will match for (“error”, code, 100) and(“error”, code, 800).

Guard

We can add anif clause to a pattern, known as a «guard». If theguard is false,match goes on to try the next case block. Notethat value capture happens before the guard is evaluated:

matchpoint:casePoint(x,y)ifx==y:print(f"The point is located on the diagonal Y=X at{x}.")casePoint(x,y):print(f"Point is not on the diagonal.")

Other Key Features

Several other key features:

  • Like unpacking assignments, tuple and list patterns have exactly thesame meaning and actually match arbitrary sequences. Technically,the subject must be a sequence.Therefore, an important exception is that patterns don’t match iterators.Also, to prevent a common mistake, sequence patterns don’t match strings.

  • Sequence patterns support wildcards:[x,y,*rest] and(x,y,*rest) work similar to wildcards in unpacking assignments. Thename after* may also be_, so(x,y,*_) matches a sequenceof at least two items without binding the remaining items.

  • Mapping patterns:{"bandwidth":b,"latency":l} captures the"bandwidth" and"latency" values from a dict. Unlike sequencepatterns, extra keys are ignored. A wildcard**rest is alsosupported. (But**_ would be redundant, so is not allowed.)

  • Subpatterns may be captured using theas keyword:

    case(Point(x1,y1),Point(x2,y2)asp2):...

    This binds x1, y1, x2, y2 like you would expect without theas clause,and p2 to the entire second item of the subject.

  • Most literals are compared by equality. However, the singletonsTrue,False andNone are compared by identity.

  • Named constants may be used in patterns. These named constants must bedotted names to prevent the constant from being interpreted as a capturevariable:

    fromenumimportEnumclassColor(Enum):RED=0GREEN=1BLUE=2color=Color.GREENmatchcolor:caseColor.RED:print("I see red!")caseColor.GREEN:print("Grass is green")caseColor.BLUE:print("I'm feeling the blues :(")

For the full specification seePEP 634. Motivation and rationaleare inPEP 635, and a longer tutorial is inPEP 636.

OptionalEncodingWarning andencoding="locale" option

The default encoding ofTextIOWrapper andopen() isplatform and locale dependent. Since UTF-8 is used on most Unixplatforms, omittingencoding option when opening UTF-8 files(e.g. JSON, YAML, TOML, Markdown) is a very common bug. For example:

# BUG: "rb" mode or encoding="utf-8" should be used.withopen("data.json")asf:data=json.load(f)

To find this type of bug, an optionalEncodingWarning is added.It is emitted whensys.flags.warn_default_encodingis true and locale-specific default encoding is used.

-Xwarn_default_encoding option andPYTHONWARNDEFAULTENCODINGare added to enable the warning.

SeeText Encoding for more information.

New Features Related to Type Hints

This section covers major changes affectingPEP 484 type hints andthetyping module.

PEP 604: New Type Union Operator

A new type union operator was introduced which enables the syntaxX|Y.This provides a cleaner way of expressing “either type X or type Y” instead ofusingtyping.Union, especially in type hints.

In previous versions of Python, to apply a type hint for functions acceptingarguments of multiple types,typing.Union was used:

defsquare(number:Union[int,float])->Union[int,float]:returnnumber**2

Type hints can now be written in a more succinct manner:

defsquare(number:int|float)->int|float:returnnumber**2

This new syntax is also accepted as the second argument toisinstance()andissubclass():

>>>isinstance(1,int|str)True

SeeΤύπος Ένωσης andPEP 604 for more details.

(Contributed by Maggie Moss and Philippe Prados inbpo-41428,with additions by Yurii Karabas and Serhiy Storchaka inbpo-44490.)

PEP 612: Parameter Specification Variables

Two new options to improve the information provided to static type checkers forPEP 484“sCallable have been added to thetyping module.

The first is the parameter specification variable. They are used to forward theparameter types of one callable to another callable – a pattern commonlyfound in higher order functions and decorators. Examples of usage can be foundintyping.ParamSpec. Previously, there was no easy way to type annotatedependency of parameter types in such a precise manner.

The second option is the newConcatenate operator. It’s used in conjunctionwith parameter specification variables to type annotate a higher order callablewhich adds or removes parameters of another callable. Examples of usage canbe found intyping.Concatenate.

Seetyping.Callable,typing.ParamSpec,typing.Concatenate,typing.ParamSpecArgs,typing.ParamSpecKwargs, andPEP 612 for more details.

(Contributed by Ken Jin inbpo-41559, with minor enhancements by JelleZijlstra inbpo-43783. PEP written by Mark Mendoza.)

PEP 613: TypeAlias

PEP 484 introduced the concept of type aliases, only requiring them to betop-level unannotated assignments. This simplicity sometimes made it difficultfor type checkers to distinguish between type aliases and ordinary assignments,especially when forward references or invalid types were involved. Compare:

StrCache='Cache[str]'# a type aliasLOG_PREFIX='LOG[DEBUG]'# a module constant

Now thetyping module has a special valueTypeAliaswhich lets you declare type aliases more explicitly:

StrCache:TypeAlias='Cache[str]'# a type aliasLOG_PREFIX='LOG[DEBUG]'# a module constant

SeePEP 613 for more details.

(Contributed by Mikhail Golubev inbpo-41923.)

PEP 647: User-Defined Type Guards

TypeGuard has been added to thetyping module to annotatetype guard functions and improve information provided to static type checkersduring type narrowing. For more information, please seeTypeGuard“s documentation, andPEP 647.

(Contributed by Ken Jin and Guido van Rossum inbpo-43766.PEP written by Eric Traut.)

Other Language Changes

  • Theint type has a new methodint.bit_count(), returning thenumber of ones in the binary expansion of a given integer, also knownas the population count. (Contributed by Niklas Fiekas inbpo-29882.)

  • The views returned bydict.keys(),dict.values() anddict.items() now all have amapping attribute that gives atypes.MappingProxyType object wrapping the originaldictionary. (Contributed by Dennis Sweeney inbpo-40890.)

  • PEP 618: Thezip() function now has an optionalstrict flag, usedto require that all the iterables have an equal length.

  • Builtin and extension functions that take integer arguments no longer acceptDecimals,Fractions and otherobjects that can be converted to integers only with a loss (e.g. that havethe__int__() method but do not have the__index__() method).(Contributed by Serhiy Storchaka inbpo-37999.)

  • Ifobject.__ipow__() returnsNotImplemented, the operator willcorrectly fall back toobject.__pow__() andobject.__rpow__() as expected.(Contributed by Alex Shkop inbpo-38302.)

  • Assignment expressions can now be used unparenthesized within set literalsand set comprehensions, as well as in sequence indexes (but not slices).

  • Functions have a new__builtins__ attribute which is used to look forbuiltin symbols when a function is executed, instead of looking into__globals__['__builtins__']. The attribute is initialized from__globals__["__builtins__"] if it exists, else from the current builtins.(Contributed by Mark Shannon inbpo-42990.)

  • Two new builtin functions –aiter() andanext() have been addedto provide asynchronous counterparts toiter() andnext(),respectively.(Contributed by Joshua Bronson, Daniel Pope, and Justin Wang inbpo-31861.)

  • Static methods (@staticmethod) and class methods(@classmethod) now inherit the method attributes(__module__,__name__,__qualname__,__doc__,__annotations__) and have a new__wrapped__ attribute.Moreover, static methods are now callable as regular functions.(Contributed by Victor Stinner inbpo-43682.)

  • Annotations for complex targets (everything besidesimplename targetsdefined byPEP 526) no longer cause any runtime effects withfrom__future__importannotations.(Contributed by Batuhan Taskaya inbpo-42737.)

  • Class and module objects now lazy-create empty annotations dicts on demand.The annotations dicts are stored in the object’s__dict__ forbackwards compatibility. This improves the best practices for workingwith__annotations__; for more information, please seeAnnotations Best Practices.(Contributed by Larry Hastings inbpo-43901.)

  • Annotations consist ofyield,yieldfrom,await or named expressionsare now forbidden underfrom__future__importannotations due to their sideeffects.(Contributed by Batuhan Taskaya inbpo-42725.)

  • Usage of unbound variables,super() and other expressions that mightalter the processing of symbol table as annotations are now renderedeffectless underfrom__future__importannotations.(Contributed by Batuhan Taskaya inbpo-42725.)

  • Hashes of NaN values of bothfloat type anddecimal.Decimal type now depend on object identity. Formerly, theyalways hashed to0 even though NaN values are not equal to one another.This caused potentially quadratic runtime behavior due to excessive hashcollisions when creating dictionaries and sets containing multiple NaNs.(Contributed by Raymond Hettinger inbpo-43475.)

  • ASyntaxError (instead of aNameError) will be raised when deletingthe__debug__ constant. (Contributed by Donghee Na inbpo-45000.)

  • SyntaxError exceptions now haveend_lineno andend_offset attributes. They will beNone if not determined.(Contributed by Pablo Galindo inbpo-43914.)

New Modules

  • None.

Improved Modules

asyncio

Add missingconnect_accepted_socket()method.(Contributed by Alex Grönholm inbpo-41332.)

argparse

Misleading phrase «optional arguments» was replaced with «options» in argparse help. Some tests might require adaptation if they rely on exact output match.(Contributed by Raymond Hettinger inbpo-9694.)

array

Theindex() method ofarray.array now hasoptionalstart andstop parameters.(Contributed by Anders Lorentsen and Zackery Spytz inbpo-31956.)

asynchat, asyncore, smtpd

These modules have been marked as deprecated in their module documentationsince Python 3.6. An import-timeDeprecationWarning has now beenadded to all three of these modules.

base64

Addbase64.b32hexencode() andbase64.b32hexdecode() to support theBase32 Encoding with Extended Hex Alphabet.

bdb

AddclearBreakpoints() to reset all set breakpoints.(Contributed by Irit Katriel inbpo-24160.)

bisect

Added the possibility of providing akey function to the APIs in thebisectmodule. (Contributed by Raymond Hettinger inbpo-4356.)

codecs

Add acodecs.unregister() function to unregister a codec search function.(Contributed by Hai Shi inbpo-41842.)

collections.abc

The__args__ of theparameterized generic forcollections.abc.Callable are now consistent withtyping.Callable.collections.abc.Callable generic now flattens type parameters, similarto whattyping.Callable currently does. This means thatcollections.abc.Callable[[int,str],str] will have__args__ of(int,str,str); previously this was([int,str],str). To allow thischange,types.GenericAlias can now be subclassed, and a subclass willbe returned when subscripting thecollections.abc.Callable type. Notethat aTypeError may be raised for invalid forms of parameterizingcollections.abc.Callable which may have passed silently in Python 3.9.(Contributed by Ken Jin inbpo-42195.)

contextlib

Add acontextlib.aclosing() context manager to safely close async generatorsand objects representing asynchronously released resources.(Contributed by Joongi Kim and John Belmonte inbpo-41229.)

Add asynchronous context manager support tocontextlib.nullcontext().(Contributed by Tom Gringauz inbpo-41543.)

AddAsyncContextDecorator, for supporting usage of asynccontext managers as decorators.

curses

The extended color functions added in ncurses 6.1 will be used transparentlybycurses.color_content(),curses.init_color(),curses.init_pair(), andcurses.pair_content(). A new function,curses.has_extended_color_support(), indicates whether extended colorsupport is provided by the underlying ncurses library.(Contributed by Jeffrey Kintscher and Hans Petter Jansson inbpo-36982.)

TheBUTTON5_* constants are now exposed in thecurses module ifthey are provided by the underlying curses library.(Contributed by Zackery Spytz inbpo-39273.)

dataclasses

__slots__

Addedslots parameter indataclasses.dataclass() decorator.(Contributed by Yurii Karabas inbpo-42269)

Keyword-only fields

dataclasses now supports fields that are keyword-only in thegenerated __init__ method. There are a number of ways of specifyingkeyword-only fields.

You can say that every field is keyword-only:

fromdataclassesimportdataclass@dataclass(kw_only=True)classBirthday:name:strbirthday:datetime.date

Bothname andbirthday are keyword-only parameters to thegenerated __init__ method.

You can specify keyword-only on a per-field basis:

fromdataclassesimportdataclass,field@dataclassclassBirthday:name:strbirthday:datetime.date=field(kw_only=True)

Here onlybirthday is keyword-only. If you setkw_only onindividual fields, be aware that there are rules about re-orderingfields due to keyword-only fields needing to follow non-keyword-onlyfields. See the full dataclasses documentation for details.

You can also specify that all fields following a KW_ONLY marker arekeyword-only. This will probably be the most common usage:

fromdataclassesimportdataclass,KW_ONLY@dataclassclassPoint:x:floaty:float_:KW_ONLYz:float=0.0t:float=0.0

Here,z andt are keyword-only parameters, whilex andy are not.(Contributed by Eric V. Smith inbpo-43532.)

distutils

The entiredistutils package is deprecated, to be removed in Python3.12. Its functionality for specifying package builds has already beencompletely replaced by third-party packagessetuptools andpackaging, and most other commonly used APIs are available elsewherein the standard library (such asplatform,shutil,subprocess orsysconfig). There are no plans to migrateany other functionality fromdistutils, and applications that areusing other functions should plan to make private copies of the code.Refer toPEP 632 for discussion.

Thebdist_wininst command deprecated in Python 3.8 has been removed.Thebdist_wheel command is now recommended to distribute binary packageson Windows.(Contributed by Victor Stinner inbpo-42802.)

doctest

When a module does not define__loader__, fall back to__spec__.loader.(Contributed by Brett Cannon inbpo-42133.)

encodings

encodings.normalize_encoding() now ignores non-ASCII characters.(Contributed by Hai Shi inbpo-39337.)

enum

Enum__repr__() now returnsenum_name.member_name and__str__() now returnsmember_name. Stdlib enums available asmodule constants have arepr() ofmodule_name.member_name.(Contributed by Ethan Furman inbpo-40066.)

Addenum.StrEnum for enums where all members are strings.(Contributed by Ethan Furman inbpo-41816.)

fileinput

Addencoding anderrors parameters infileinput.input() andfileinput.FileInput.(Contributed by Inada Naoki inbpo-43712.)

fileinput.hook_compressed() now returnsTextIOWrapper objectwhenmode is «r» and file is compressed, like uncompressed files.(Contributed by Inada Naoki inbpo-5758.)

faulthandler

Thefaulthandler module now detects if a fatal error occurs during agarbage collector collection.(Contributed by Victor Stinner inbpo-44466.)

gc

Add audit hooks forgc.get_objects(),gc.get_referrers() andgc.get_referents(). (Contributed by Pablo Galindo inbpo-43439.)

glob

Add theroot_dir anddir_fd parameters inglob() andiglob() which allow to specify the root directory for searching.(Contributed by Serhiy Storchaka inbpo-38144.)

hashlib

The hashlib module requires OpenSSL 1.1.1 or newer.(Contributed by Christian Heimes inPEP 644 andbpo-43669.)

The hashlib module has preliminary support for OpenSSL 3.0.0.(Contributed by Christian Heimes inbpo-38820 and other issues.)

The pure-Python fallback ofpbkdf2_hmac() is deprecated. Inthe future PBKDF2-HMAC will only be available when Python has been built withOpenSSL support.(Contributed by Christian Heimes inbpo-43880.)

hmac

The hmac module now uses OpenSSL’s HMAC implementation internally.(Contributed by Christian Heimes inbpo-40645.)

IDLE and idlelib

Make IDLE invokesys.excepthook() (when started without “-n”).User hooks were previously ignored. (Contributed by Ken Hilton inbpo-43008.)

Rearrange the settings dialog. Split the General tab into Windowsand Shell/Ed tabs. Move help sources, which extend the Help menu, to theExtensions tab. Make space for new options and shorten the dialog. Thelatter makes the dialog better fit small screens. (Contributed by Terry JanReedy inbpo-40468.) Move the indent space setting from the Font tab tothe new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy inbpo-33962.)

The changes above were backported to a 3.9 maintenance release.

Add a Shell sidebar. Move the primary prompt (“>>>”) to the sidebar.Add secondary prompts (”…”) to the sidebar. Left click and optionaldrag selects one or more lines of text, as with the editorline number sidebar. Right click after selecting text lines displaysa context menu with “copy with prompts”. This zips together promptsfrom the sidebar with lines from the selected text. This option alsoappears on the context menu for the text. (Contributed by Tal Einatinbpo-37903.)

Use spaces instead of tabs to indent interactive code. This makesinteractive code entries “look right”. Making this feasible was amajor motivation for adding the shell sidebar. (Contributed byTerry Jan Reedy inbpo-37892.)

Highlight the newsoft keywordsmatch,case, and_ inpattern-matching statements. However, this highlighting is not perfectand will be incorrect in some rare cases, including some_-s incase patterns. (Contributed by Tal Einat inbpo-44010.)

New in 3.10 maintenance releases.

Apply syntax highlighting to.pyi files. (Contributed by AlexWaygood and Terry Jan Reedy inbpo-45447.)

Include prompts when saving Shell with inputs and outputs.(Contributed by Terry Jan Reedy ingh-95191.)

importlib.metadata

Feature parity withimportlib_metadata 4.6(history).

importlib.metadata entry pointsnow provide a nicer experiencefor selecting entry points by group and name through a newimportlib.metadata.EntryPoints class. See the CompatibilityNote in the docs for more info on the deprecation and usage.

Addedimportlib.metadata.packages_distributions()for resolving top-level Python modules and packages to theirimportlib.metadata.Distribution.

inspect

When a module does not define__loader__, fall back to__spec__.loader.(Contributed by Brett Cannon inbpo-42133.)

Addinspect.get_annotations(), which safely computes the annotationsdefined on an object. It works around the quirks of accessing the annotationson various types of objects, and makes very few assumptions about the objectit examines.inspect.get_annotations() can also correctly un-stringizestringized annotations.inspect.get_annotations() is now consideredbest practice for accessing the annotations dict defined on any Python object;for more information on best practices for working with annotations, please seeAnnotations Best Practices.Relatedly,inspect.signature(),inspect.Signature.from_callable(), andinspect.Signature.from_function()now callinspect.get_annotations() to retrieve annotations. This meansinspect.signature() andinspect.Signature.from_callable() canalso now un-stringize stringized annotations.(Contributed by Larry Hastings inbpo-43817.)

itertools

Additertools.pairwise().(Contributed by Raymond Hettinger inbpo-38200.)

linecache

When a module does not define__loader__, fall back to__spec__.loader.(Contributed by Brett Cannon inbpo-42133.)

os

Addos.cpu_count() support for VxWorks RTOS.(Contributed by Peixing Xin inbpo-41440.)

Add a new functionos.eventfd() and related helpers to wrap theeventfd2 syscall on Linux.(Contributed by Christian Heimes inbpo-41001.)

Addos.splice() that allows to move data between two filedescriptors without copying between kernel address space and useraddress space, where one of the file descriptors must refer to apipe. (Contributed by Pablo Galindo inbpo-41625.)

AddO_EVTONLY,O_FSYNC,O_SYMLINKandO_NOFOLLOW_ANY for macOS.(Contributed by Donghee Na inbpo-43106.)

os.path

os.path.realpath() now accepts astrict keyword-only argument. When settoTrue,OSError is raised if a path doesn’t exist or a symlink loopis encountered.(Contributed by Barney Gale inbpo-43757.)

pathlib

Add slice support toPurePath.parents.(Contributed by Joshua Cannon inbpo-35498.)

Add negative indexing support toPurePath.parents.(Contributed by Yaroslav Pankovych inbpo-21041.)

AddPath.hardlink_to method thatsupersedeslink_to(). The new method has the same argumentorder assymlink_to().(Contributed by Barney Gale inbpo-39950.)

pathlib.Path.stat() andchmod() now accept afollow_symlinks keyword-only argument for consistency with correspondingfunctions in theos module.(Contributed by Barney Gale inbpo-39906.)

platform

Addplatform.freedesktop_os_release() to retrieve operation systemidentification fromfreedesktop.org os-release standard file.(Contributed by Christian Heimes inbpo-28468.)

pprint

pprint.pprint() now accepts a newunderscore_numbers keyword argument.(Contributed by sblondon inbpo-42914.)

pprint can now pretty-printdataclasses.dataclass instances.(Contributed by Lewis Gaul inbpo-43080.)

py_compile

Add--quiet option to command-line interface ofpy_compile.(Contributed by Gregory Schevchenko inbpo-38731.)

pyclbr

Add anend_lineno attribute to theFunction andClassobjects in the tree returned bypyclbr.readmodule() andpyclbr.readmodule_ex(). It matches the existing (start)lineno.(Contributed by Aviral Srivastava inbpo-38307.)

shelve

Theshelve module now usespickle.DEFAULT_PROTOCOL by defaultinstead ofpickle protocol3 when creating shelves.(Contributed by Zackery Spytz inbpo-34204.)

statistics

Addcovariance(), Pearson’scorrelation(), and simplelinear_regression() functions.(Contributed by Tymoteusz Wołodźko inbpo-38490.)

site

When a module does not define__loader__, fall back to__spec__.loader.(Contributed by Brett Cannon inbpo-42133.)

socket

The exceptionsocket.timeout is now an alias ofTimeoutError.(Contributed by Christian Heimes inbpo-42413.)

Add option to create MPTCP sockets withIPPROTO_MPTCP(Contributed by Rui Cunha inbpo-43571.)

AddIP_RECVTOS option to receive the type of service (ToS) or DSCP/ECN fields(Contributed by Georg Sauthoff inbpo-44077.)

ssl

The ssl module requires OpenSSL 1.1.1 or newer.(Contributed by Christian Heimes inPEP 644 andbpo-43669.)

The ssl module has preliminary support for OpenSSL 3.0.0 and new optionOP_IGNORE_UNEXPECTED_EOF.(Contributed by Christian Heimes inbpo-38820,bpo-43794,bpo-43788,bpo-43791,bpo-43799,bpo-43920,bpo-43789, andbpo-43811.)

Deprecated function and use of deprecated constants now result inaDeprecationWarning.ssl.SSLContext.options hasOP_NO_SSLv2 andOP_NO_SSLv3 set by default andtherefore cannot warn about setting the flag again. Thedeprecation section has a list of deprecatedfeatures.(Contributed by Christian Heimes inbpo-43880.)

The ssl module now has more secure default settings. Ciphers without forwardsecrecy or SHA-1 MAC are disabled by default. Security level 2 prohibitsweak RSA, DH, and ECC keys with less than 112 bits of security.SSLContext defaults to minimum protocol version TLS 1.2.Settings are based on Hynek Schlawack’s research.(Contributed by Christian Heimes inbpo-43998.)

The deprecated protocols SSL 3.0, TLS 1.0, and TLS 1.1 are no longerofficially supported. Python does not block them actively. HoweverOpenSSL build options, distro configurations, vendor patches, and ciphersuites may prevent a successful handshake.

Add atimeout parameter to thessl.get_server_certificate() function.(Contributed by Zackery Spytz inbpo-31870.)

The ssl module uses heap-types and multi-phase initialization.(Contributed by Christian Heimes inbpo-42333.)

A new verify flagVERIFY_X509_PARTIAL_CHAIN has been added.(Contributed by l0x inbpo-40849.)

sqlite3

Add audit events forconnect/handle(),enable_load_extension(), andload_extension().(Contributed by Erlend E. Aasland inbpo-43762.)

sys

Addsys.orig_argv attribute: the list of the original command linearguments passed to the Python executable.(Contributed by Victor Stinner inbpo-23427.)

Addsys.stdlib_module_names, containing the list of the standard librarymodule names.(Contributed by Victor Stinner inbpo-42955.)

_thread

_thread.interrupt_main() now takes an optional signal number tosimulate (the default is stillsignal.SIGINT).(Contributed by Antoine Pitrou inbpo-43356.)

threading

Addthreading.gettrace() andthreading.getprofile() toretrieve the functions set bythreading.settrace() andthreading.setprofile() respectively.(Contributed by Mario Corchero inbpo-42251.)

Addthreading.__excepthook__ to allow retrieving the original valueofthreading.excepthook() in case it is set to a broken or a differentvalue.(Contributed by Mario Corchero inbpo-42308.)

traceback

Theformat_exception(),format_exception_only(), andprint_exception() functions can now take an exception objectas a positional-only argument.(Contributed by Zackery Spytz and Matthias Bussonnier inbpo-26389.)

types

Reintroduce thetypes.EllipsisType,types.NoneTypeandtypes.NotImplementedType classes, providing a new setof types readily interpretable by type checkers.(Contributed by Bas van Beek inbpo-41810.)

typing

For major changes, seeNew Features Related to Type Hints.

The behavior oftyping.Literal was changed to conform withPEP 586and to match the behavior of static type checkers specified in the PEP.

  1. Literal now de-duplicates parameters.

  2. Equality comparisons betweenLiteral objects are now order independent.

  3. Literal comparisons now respect types. For example,Literal[0]==Literal[False] previously evaluated toTrue. It isnowFalse. To support this change, the internally used type cache nowsupports differentiating types.

  4. Literal objects will now raise aTypeError exception duringequality comparisons if any of their parameters are nothashable.Note that declaringLiteral with unhashable parameters will not throwan error:

    >>>fromtypingimportLiteral>>>Literal[{0}]>>>Literal[{0}]==Literal[{False}]Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:unhashable type: 'set'

(Contributed by Yurii Karabas inbpo-42345.)

Add new functiontyping.is_typeddict() to introspect if an annotationis atyping.TypedDict.(Contributed by Patrick Reader inbpo-41792.)

Subclasses oftyping.Protocol which only have data variables declaredwill now raise aTypeError when checked withisinstance unless theyare decorated withruntime_checkable(). Previously, these checkspassed silently. Users should decorate theirsubclasses with theruntime_checkable() decoratorif they want runtime protocols.(Contributed by Yurii Karabas inbpo-38908.)

Importing from thetyping.io andtyping.re submodules will now emitDeprecationWarning. These submodules have been deprecated sincePython 3.8 and will be removed in a future version of Python. Anythingbelonging to those submodules should be imported directly fromtyping instead.(Contributed by Sebastian Rittau inbpo-38291.)

unittest

Add new methodassertNoLogs() to complement theexistingassertLogs(). (Contributed by Kit Yan Choiinbpo-39385.)

urllib.parse

Python versions earlier than Python 3.10 allowed using both; and& asquery parameter separators inurllib.parse.parse_qs() andurllib.parse.parse_qsl(). Due to security concerns, and to conform withnewer W3C recommendations, this has been changed to allow only a singleseparator key, with& as the default. This change also affectscgi.parse() andcgi.parse_multipart() as they use the affectedfunctions internally. For more details, please see their respectivedocumentation.(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin inbpo-42967.)

The presence of newline or tab characters in parts of a URL allows for someforms of attacks. Following the WHATWG specification that updatesRFC 3986,ASCII newline\n,\r and tab\t characters are stripped from theURL by the parser inurllib.parse preventing such attacks. The removalcharacters are controlled by a new module level variableurllib.parse._UNSAFE_URL_BYTES_TO_REMOVE. (Seegh-88048)

xml

Add aLexicalHandler class to thexml.sax.handler module.(Contributed by Jonathan Gossage and Zackery Spytz inbpo-35018.)

zipimport

Add methods related toPEP 451:find_spec(),zipimport.zipimporter.create_module(), andzipimport.zipimporter.exec_module().(Contributed by Brett Cannon inbpo-42131.)

Addinvalidate_caches() method.(Contributed by Desmond Cheong inbpo-14678.)

Optimizations

  • Constructorsstr(),bytes() andbytearray() are now faster(around 30–40% for small objects).(Contributed by Serhiy Storchaka inbpo-41334.)

  • Therunpy module now imports fewer modules.Thepython3-mmodule-name command startup time is 1.4x faster inaverage. On Linux,python3-I-mmodule-name imports 69 modules on Python3.9, whereas it only imports 51 modules (-18) on Python 3.10.(Contributed by Victor Stinner inbpo-41006 andbpo-41718.)

  • TheLOAD_ATTR instruction now uses new «per opcode cache» mechanism. Itis about 36% faster now for regular attributes and 44% faster for slots.(Contributed by Pablo Galindo and Yury Selivanov inbpo-42093 and Guidovan Rossum inbpo-42927, based on ideas implemented originally in PyPyand MicroPython.)

  • When building Python with--enable-optimizations now-fno-semantic-interposition is added to both the compile and link line.This speeds builds of the Python interpreter created with--enable-sharedwithgcc by up to 30%. Seethis articlefor more details. (Contributed by Victor Stinner and Pablo Galindo inbpo-38980.)

  • Use a new output buffer management code forbz2 /lzma /zlib modules, and add.readall() function to_compression.DecompressReader class. bz2 decompression is now 1.09x ~ 1.17xfaster, lzma decompression 1.20x ~ 1.32x faster,GzipFile.read(-1) 1.11x~ 1.18x faster. (Contributed by Ma Lin, reviewed by Gregory P. Smith, inbpo-41486)

  • When using stringized annotations, annotations dicts for functions are no longercreated when the function is created. Instead, they are stored as a tuple ofstrings, and the function object lazily converts this into the annotations dicton demand. This optimization cuts the CPU time needed to define an annotatedfunction by half.(Contributed by Yurii Karabas and Inada Naoki inbpo-42202.)

  • Substring search functions such asstr1instr2 andstr2.find(str1)now sometimes use Crochemore & Perrin’s «Two-Way» string searchingalgorithm to avoid quadratic behavior on long strings. (Contributedby Dennis Sweeney inbpo-41972)

  • Add micro-optimizations to_PyType_Lookup() to improve type attribute cache lookupperformance in the common case of cache hits. This makes the interpreter 1.04 times fasteron average. (Contributed by Dino Viehland inbpo-43452.)

  • The following built-in functions now support the fasterPEP 590 vectorcall calling convention:map(),filter(),reversed(),bool() andfloat().(Contributed by Donghee Na and Jeroen Demeyer inbpo-43575,bpo-43287,bpo-41922,bpo-41873 andbpo-41870.)

  • BZ2File performance is improved by removing internalRLock.This makesBZ2File thread unsafe in the face of multiple simultaneousreaders or writers, just like its equivalent classes ingzip andlzma have always been. (Contributed by Inada Naoki inbpo-43785.)

Deprecated

  • Currently Python accepts numeric literals immediately followed by keywords,for example0inx,1orx,0if1else2. It allows confusingand ambiguous expressions like[0x1forxiny] (which can beinterpreted as[0x1forxiny] or[0x1forxiny]). Starting inthis release, a deprecation warning is raised if the numeric literal isimmediately followed by one of keywordsand,else,for,if,in,is andor.In future releases it will be changed to syntax warning, and finally tosyntax error.(Contributed by Serhiy Storchaka inbpo-43833.)

  • Starting in this release, there will be a concerted effort to begincleaning up old import semantics that were kept for Python 2.7compatibility. Specifically,find_loader()/find_module()(superseded byfind_spec()),load_module()(superseded byexec_module()),module_repr() (which the import systemtakes care of for you), the__package__ attribute(superseded by__spec__.parent), the__loader__ attribute(superseded by__spec__.loader), and the__cached__ attribute(superseded by__spec__.cached) will slowly be removed (as wellas other classes and methods inimportlib).ImportWarning and/orDeprecationWarning will be raisedas appropriate to help identify code which needs updating duringthis transition.

  • The entiredistutils namespace is deprecated, to be removed inPython 3.12. Refer to themodule changessection for more information.

  • Non-integer arguments torandom.randrange() are deprecated.TheValueError is deprecated in favor of aTypeError.(Contributed by Serhiy Storchaka and Raymond Hettinger inbpo-37319.)

  • The variousload_module() methods ofimportlib have beendocumented as deprecated since Python 3.6, but will now also triggeraDeprecationWarning. Useexec_module() instead.(Contributed by Brett Cannon inbpo-26131.)

  • zimport.zipimporter.load_module() has been deprecated inpreference forexec_module().(Contributed by Brett Cannon inbpo-26131.)

  • The use ofload_module() by the importsystem now triggers anImportWarning asexec_module() is preferred.(Contributed by Brett Cannon inbpo-26131.)

  • The use ofimportlib.abc.MetaPathFinder.find_module() andimportlib.abc.PathEntryFinder.find_module() by the import system nowtrigger anImportWarning asimportlib.abc.MetaPathFinder.find_spec() andimportlib.abc.PathEntryFinder.find_spec()are preferred, respectively. You can useimportlib.util.spec_from_loader() to help in porting.(Contributed by Brett Cannon inbpo-42134.)

  • The use ofimportlib.abc.PathEntryFinder.find_loader() by the importsystem now triggers anImportWarning asimportlib.abc.PathEntryFinder.find_spec() is preferred. You can useimportlib.util.spec_from_loader() to help in porting.(Contributed by Brett Cannon inbpo-43672.)

  • The various implementations ofimportlib.abc.MetaPathFinder.find_module() (importlib.machinery.BuiltinImporter.find_module(),importlib.machinery.FrozenImporter.find_module(),importlib.machinery.WindowsRegistryFinder.find_module(),importlib.machinery.PathFinder.find_module(),importlib.abc.MetaPathFinder.find_module() ),importlib.abc.PathEntryFinder.find_module() (importlib.machinery.FileFinder.find_module() ), andimportlib.abc.PathEntryFinder.find_loader() (importlib.machinery.FileFinder.find_loader() )now raiseDeprecationWarning and are slated for removal inPython 3.12 (previously they were documented as deprecated in Python 3.4).(Contributed by Brett Cannon inbpo-42135.)

  • importlib.abc.Finder is deprecated (including its sole method,find_module()). Bothimportlib.abc.MetaPathFinder andimportlib.abc.PathEntryFinderno longer inherit from the class. Users should inherit from one of these twoclasses as appropriate instead.(Contributed by Brett Cannon inbpo-42135.)

  • The deprecations ofimp,importlib.find_loader(),importlib.util.set_package_wrapper(),importlib.util.set_loader_wrapper(),importlib.util.module_for_loader(),pkgutil.ImpImporter, andpkgutil.ImpLoader have all been updated to list Python 3.12 as theslated version of removal (they began raisingDeprecationWarning inprevious versions of Python).(Contributed by Brett Cannon inbpo-43720.)

  • The import system now uses the__spec__ attribute on modules beforefalling back onmodule_repr() for a module’s__repr__() method. Removal of the use ofmodule_repr() is scheduledfor Python 3.12.(Contributed by Brett Cannon inbpo-42137.)

  • importlib.abc.Loader.module_repr(),importlib.machinery.FrozenLoader.module_repr(), andimportlib.machinery.BuiltinLoader.module_repr() are deprecated andslated for removal in Python 3.12.(Contributed by Brett Cannon inbpo-42136.)

  • sqlite3.OptimizedUnicode has been undocumented and obsolete since Python3.3, when it was made an alias tostr. It is now deprecated,scheduled for removal in Python 3.12.(Contributed by Erlend E. Aasland inbpo-42264.)

  • The undocumented built-in functionsqlite3.enable_shared_cache is nowdeprecated, scheduled for removal in Python 3.12. Its use is stronglydiscouraged by the SQLite3 documentation. Seethe SQLite3 docs for more details.If a shared cache must be used, open the database in URI mode using thecache=shared query parameter.(Contributed by Erlend E. Aasland inbpo-24464.)

  • The followingthreading methods are now deprecated:

    (Contributed by Jelle Zijlstra ingh-87889.)

  • pathlib.Path.link_to() is deprecated and slated for removal inPython 3.12. Usepathlib.Path.hardlink_to() instead.(Contributed by Barney Gale inbpo-39950.)

  • cgi.log() is deprecated and slated for removal in Python 3.12.(Contributed by Inada Naoki inbpo-41139.)

  • The followingssl features have been deprecated since Python 3.6,Python 3.7, or OpenSSL 1.1.0 and will be removed in 3.11:

  • The threading debug (PYTHONTHREADDEBUG environment variable) isdeprecated in Python 3.10 and will be removed in Python 3.12. This featurerequires adebug build of Python.(Contributed by Victor Stinner inbpo-44584.)

  • Importing from thetyping.io andtyping.re submodules will now emitDeprecationWarning. These submodules will be removed in a future versionof Python. Anything belonging to these submodules should be imported directlyfromtyping instead.(Contributed by Sebastian Rittau inbpo-38291.)

Removed

  • Removed special methods__int__,__float__,__floordiv__,__mod__,__divmod__,__rfloordiv__,__rmod__ and__rdivmod__ of thecomplex class. They always raisedaTypeError.(Contributed by Serhiy Storchaka inbpo-41974.)

  • TheParserBase.error() method from the private and undocumented_markupbasemodule has been removed.html.parser.HTMLParser is the only subclass ofParserBase and itserror() implementation was already removed inPython 3.5.(Contributed by Berker Peksag inbpo-31844.)

  • Removed theunicodedata.ucnhash_CAPI attribute which was an internalPyCapsule object. The related private_PyUnicode_Name_CAPI structure wasmoved to the internal C API.(Contributed by Victor Stinner inbpo-42157.)

  • Removed theparser module, which was deprecated in 3.9 due to theswitch to the new PEG parser, as well as all the C source and header filesthat were only being used by the old parser, includingnode.h,parser.h,graminit.h andgrammar.h.

  • Removed the Public C API functionsPyParser_SimpleParseStringFlags,PyParser_SimpleParseStringFlagsFilename,PyParser_SimpleParseFileFlags andPyNode_Compilethat were deprecated in 3.9 due to the switch to the new PEG parser.

  • Removed theformatter module, which was deprecated in Python 3.4.It is somewhat obsolete, little used, and not tested. It was originallyscheduled to be removed in Python 3.6, but such removals were delayed untilafter Python 2.7 EOL. Existing users should copy whatever classes they useinto their code.(Contributed by Donghee Na and Terry J. Reedy inbpo-42299.)

  • Removed thePyModule_GetWarningsModule() function that was uselessnow due to the_warnings module was converted to a builtin module in 2.6.(Contributed by Hai Shi inbpo-42599.)

  • Remove deprecated aliases toΑφηρημένες Βασικές Κλάσεις Συλλογών fromthecollections module.(Contributed by Victor Stinner inbpo-37324.)

  • Theloop parameter has been removed from most ofasyncio“shigh-level API following deprecationin Python 3.8. The motivation behind this change is multifold:

    1. This simplifies the high-level API.

    2. The functions in the high-level API have been implicitly getting thecurrent thread’s running event loop since Python 3.7. There isn’t a need topass the event loop to the API in most normal use cases.

    3. Event loop passing is error-prone especially when dealing with loopsrunning in different threads.

    Note that the low-level API will still acceptloop.SeeChanges in the Python API for examples of how to replace existing code.

    (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanleyinbpo-42392.)

Porting to Python 3.10

This section lists previously described changes and other bugfixesthat may require changes to your code.

Changes in the Python syntax

  • Deprecation warning is now emitted when compiling previously valid syntaxif the numeric literal is immediately followed by a keyword (like in0inx).In future releases it will be changed to syntax warning, and finally to asyntax error. To get rid of the warning and make the code compatible withfuture releases just add a space between the numeric literal and thefollowing keyword.(Contributed by Serhiy Storchaka inbpo-43833.)

Changes in the Python API

  • Theetype parameters of theformat_exception(),format_exception_only(), andprint_exception() functions in thetraceback modulehave been renamed toexc.(Contributed by Zackery Spytz and Matthias Bussonnier inbpo-26389.)

  • atexit: At Python exit, if a callback registered withatexit.register() fails, its exception is now logged. Previously, onlysome exceptions were logged, and the last exception was always silentlyignored.(Contributed by Victor Stinner inbpo-42639.)

  • collections.abc.Callable generic now flattens type parameters, similarto whattyping.Callable currently does. This means thatcollections.abc.Callable[[int,str],str] will have__args__ of(int,str,str); previously this was([int,str],str). Code whichaccesses the arguments viatyping.get_args() or__args__ need to accountfor this change. Furthermore,TypeError may be raised for invalid formsof parameterizingcollections.abc.Callable which may have passedsilently in Python 3.9.(Contributed by Ken Jin inbpo-42195.)

  • socket.htons() andsocket.ntohs() now raiseOverflowErrorinstead ofDeprecationWarning if the given parameter will not fit ina 16-bit unsigned integer.(Contributed by Erlend E. Aasland inbpo-42393.)

  • Theloop parameter has been removed from most ofasyncio“shigh-level API following deprecationin Python 3.8.

    A coroutine that currently looks like this:

    asyncdeffoo(loop):awaitasyncio.sleep(1,loop=loop)

    Should be replaced with this:

    asyncdeffoo():awaitasyncio.sleep(1)

    Iffoo() was specifically designednot to run in the current thread’srunning event loop (e.g. running in another thread’s event loop), considerusingasyncio.run_coroutine_threadsafe() instead.

    (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanleyinbpo-42392.)

  • Thetypes.FunctionType constructor now inherits the current builtinsif theglobals dictionary has no"__builtins__" key, rather than using{"None":None} as builtins: same behavior aseval() andexec() functions. Defining a function withdeffunction(...):...in Python is not affected, globals cannot be overridden with this syntax: italso inherits the current builtins.(Contributed by Victor Stinner inbpo-42990.)

Changes in the C API

  • The C API functionsPyParser_SimpleParseStringFlags,PyParser_SimpleParseStringFlagsFilename,PyParser_SimpleParseFileFlags,PyNode_Compile and the typeused by these functions,struct_node, were removed due to the switchto the new PEG parser.

    Source should be now be compiled directly to a code object using, forexample,Py_CompileString(). The resulting code object can then beevaluated using, for example,PyEval_EvalCode().

    Specifically:

    • A call toPyParser_SimpleParseStringFlags followed byPyNode_Compile can be replaced by callingPy_CompileString().

    • There is no direct replacement forPyParser_SimpleParseFileFlags.To compile code from aFILE* argument, you will need to readthe file in C and pass the resulting buffer toPy_CompileString().

    • To compile a file given achar* filename, explicitly open the file, readit and compile the result. One way to do this is using theiomodule withPyImport_ImportModule(),PyObject_CallMethod(),PyBytes_AsString() andPy_CompileString(),as sketched below. (Declarations and error handling are omitted.)

      io_module=Import_ImportModule("io");fileobject=PyObject_CallMethod(io_module,"open","ss",filename,"rb");source_bytes_object=PyObject_CallMethod(fileobject,"read","");result=PyObject_CallMethod(fileobject,"close","");source_buf=PyBytes_AsString(source_bytes_object);code=Py_CompileString(source_buf,filename,Py_file_input);

    • ForFrameObject objects, thef_lasti member now represents a wordcodeoffset instead of a simple offset into the bytecode string. This means that thisnumber needs to be multiplied by 2 to be used with APIs that expect a byte offsetinstead (likePyCode_Addr2Line() for example). Notice as well that thef_lasti member ofFrameObject objects is not considered stable: pleaseusePyFrame_GetLineNumber() instead.

CPython bytecode changes

  • TheMAKE_FUNCTION instruction now accepts either a dict or a tuple ofstrings as the function’s annotations.(Contributed by Yurii Karabas and Inada Naoki inbpo-42202.)

Build Changes

  • PEP 644: Python now requires OpenSSL 1.1.1 or newer. OpenSSL 1.0.2 is nolonger supported.(Contributed by Christian Heimes inbpo-43669.)

  • The C99 functionssnprintf() andvsnprintf() are now requiredto build Python.(Contributed by Victor Stinner inbpo-36020.)

  • sqlite3 requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseevand Erlend E. Aasland inbpo-40744 andbpo-40810.)

  • Theatexit module must now always be built as a built-in module.(Contributed by Victor Stinner inbpo-42639.)

  • Add--disable-test-modules option to theconfigure script:don’t build nor install test modules.(Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin inbpo-27640.)

  • Add--with-wheel-pkg-dir=PATHoptionto the./configure script. Ifspecified, theensurepip module looks forsetuptools andpipwheel packages in this directory: if both are present, these wheel packagesare used instead of ensurepip bundled wheel packages.

    Some Linux distribution packaging policies recommend against bundlingdependencies. For example, Fedora installs wheel packages in the/usr/share/python-wheels/ directory and don’t install theensurepip._bundled package.

    (Contributed by Victor Stinner inbpo-42856.)

  • Add a newconfigure--without-static-libpythonoption to not build thelibpythonMAJOR.MINOR.astatic library and not install thepython.o object file.

    (Contributed by Victor Stinner inbpo-43103.)

  • Theconfigure script now uses thepkg-config utility, if available,to detect the location of Tcl/Tk headers and libraries. As before, thoselocations can be explicitly specified with the--with-tcltk-includesand--with-tcltk-libs configuration options.(Contributed by Manolis Stamatogiannakis inbpo-42603.)

  • Add--with-openssl-rpath option toconfigure script. The optionsimplifies building Python with a custom OpenSSL installation, e.g../configure--with-openssl=/path/to/openssl--with-openssl-rpath=auto.(Contributed by Christian Heimes inbpo-43466.)

C API Changes

PEP 652: Maintaining the Stable ABI

The Stable ABI (Application Binary Interface) for extension modules orembedding Python is now explicitly defined.C API Stability describes C API and ABI stability guarantees along with bestpractices for using the Stable ABI.

(Contributed by Petr Viktorin inPEP 652 andbpo-43795.)

New Features

Porting to Python 3.10

Deprecated

  • ThePyUnicode_InternImmortal() function is now deprecatedand will be removed in Python 3.12: usePyUnicode_InternInPlace()instead.(Contributed by Victor Stinner inbpo-41692.)

Removed

  • RemovedPy_UNICODE_str* functions manipulatingPy_UNICODE* strings.(Contributed by Inada Naoki inbpo-41123.)

  • RemovedPyUnicode_GetMax(). Please migrate to new (PEP 393) APIs.(Contributed by Inada Naoki inbpo-41103.)

  • RemovedPyLong_FromUnicode(). Please migrate toPyLong_FromUnicodeObject().(Contributed by Inada Naoki inbpo-41103.)

  • RemovedPyUnicode_AsUnicodeCopy(). Please usePyUnicode_AsUCS4Copy() orPyUnicode_AsWideCharString()(Contributed by Inada Naoki inbpo-41103.)

  • Removed_Py_CheckRecursionLimit variable: it has been replaced byceval.recursion_limit of thePyInterpreterState structure.(Contributed by Victor Stinner inbpo-41834.)

  • Removed undocumented macrosPy_ALLOW_RECURSION andPy_END_ALLOW_RECURSION and therecursion_critical field of thePyInterpreterState structure.(Contributed by Serhiy Storchaka inbpo-41936.)

  • Removed the undocumentedPyOS_InitInterrupts() function. InitializingPython already implicitly installs signal handlers: seePyConfig.install_signal_handlers.(Contributed by Victor Stinner inbpo-41713.)

  • Remove thePyAST_Validate() function. It is no longer possible to build aAST object (mod_ty type) with the public C API. The function was alreadyexcluded from the limited C API (PEP 384).(Contributed by Victor Stinner inbpo-43244.)

  • Remove thesymtable.h header file and the undocumented functions:

    • PyST_GetScope()

    • PySymtable_Build()

    • PySymtable_BuildObject()

    • PySymtable_Free()

    • Py_SymtableString()

    • Py_SymtableStringObject()

    ThePy_SymtableString() function was part the stable ABI by mistake butit could not be used, because thesymtable.h header file was excludedfrom the limited C API.

    Use Pythonsymtable module instead.(Contributed by Victor Stinner inbpo-43244.)

  • RemovePyOS_ReadlineFunctionPointer() from the limited C API headersand frompython3.dll, the library that provides the stable ABI onWindows. Since the function takes aFILE* argument, its ABI stabilitycannot be guaranteed.(Contributed by Petr Viktorin inbpo-43868.)

  • Removeast.h,asdl.h, andPython-ast.h header files.These functions were undocumented and excluded from the limited C API.Most names defined by these header files were not prefixed byPy and socould create names conflicts. For example,Python-ast.h defined aYield macro which was conflict with theYield name used by theWindows<winbase.h> header. Use the Pythonast module instead.(Contributed by Victor Stinner inbpo-43244.)

  • Remove the compiler and parser functions usingstruct_mod type, becausethe public AST C API was removed:

    • PyAST_Compile()

    • PyAST_CompileEx()

    • PyAST_CompileObject()

    • PyFuture_FromAST()

    • PyFuture_FromASTObject()

    • PyParser_ASTFromFile()

    • PyParser_ASTFromFileObject()

    • PyParser_ASTFromFilename()

    • PyParser_ASTFromString()

    • PyParser_ASTFromStringObject()

    These functions were undocumented and excluded from the limited C API.(Contributed by Victor Stinner inbpo-43244.)

  • Remove thepyarena.h header file with functions:

    • PyArena_New()

    • PyArena_Free()

    • PyArena_Malloc()

    • PyArena_AddPyObject()

    These functions were undocumented, excluded from the limited C API, and wereonly used internally by the compiler.(Contributed by Victor Stinner inbpo-43244.)

  • ThePyThreadState.use_tracing member has been removed to optimize Python.(Contributed by Mark Shannon inbpo-43760.)

Notable security feature in 3.10.7

Converting betweenint andstr in bases other than 2(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)now raises aValueError if the number of digits in string form isabove a limit to avoid potential denial of service attacks due to thealgorithmic complexity. This is a mitigation forCVE 2020-10735.This limit can be configured or disabled by environment variable, commandline flag, orsys APIs. See theinteger string conversionlength limitation documentation. The default limitis 4300 digits in string form.

Notable security feature in 3.10.8

The deprecatedmailcap module now refuses to inject unsafe text(filenames, MIME types, parameters) into shell commands. Instead of using suchtext, it will warn and act as if a match was not found (or for test commands,as if the test failed).(Contributed by Petr Viktorin ingh-98966.)

Notable changes in 3.10.12

tarfile

  • The extraction methods intarfile, andshutil.unpack_archive(),have a new afilter argument that allows limiting tar features than may besurprising or dangerous, such as creating files outside the destinationdirectory.SeeExtraction filters for details.In Python 3.12, use without thefilter argument will show aDeprecationWarning.In Python 3.14, the default will switch to'data'.(Contributed by Petr Viktorin inPEP 706.)