The Future for Python 2.x¶

Python 2.7 is intended to be the last major release in the 2.x series.The Python maintainers are planning to focus their future efforts onthe Python 3.x series.

This means that 2.7 will remain in place for a long time, runningproduction systems that have not been ported to Python 3.x.Two consequences of the long-term significance of 2.7 are:

• It’s very likely the 2.7 release will have a longer period ofmaintenance compared to earlier 2.x versions. Python 2.7 willcontinue to be maintained while the transition to 3.x continues, andthe developers are planning to support Python 2.7 with bug-fixreleases beyond the typical two years.

• A policy decision was made to silence warnings only of interest todevelopers.DeprecationWarning and itsdescendants are now ignored unless otherwise requested, preventingusers from seeing warnings triggered by an application. This changewas also made in the branch that will become Python 3.2. (Discussedon stdlib-sig and carried out inissue 7319.)

  In previous releases,DeprecationWarning messages wereenabled by default, providing Python developers with a clearindication of where their code may break in a future major versionof Python.

  However, there are increasingly many users of Python-basedapplications who are not directly involved in the development ofthose applications.DeprecationWarning messages areirrelevant to such users, making them worry about an applicationthat’s actually working correctly and burdening application developerswith responding to these concerns.

  You can re-enable display ofDeprecationWarning messages byrunning Python with the-Wdefault (short form:-Wd) switch, or by setting thePYTHONWARNINGSenvironment variable to"default" (or"d") before runningPython. Python code can also re-enable themby callingwarnings.simplefilter('default').

Python 3.1 Features¶

Much as Python 2.6 incorporated features from Python 3.0,version 2.7 incorporates some of the new featuresin Python 3.1. The 2.x series continues to provide toolsfor migrating to the 3.x series.

A partial list of 3.1 features that were backported to 2.7:

• The syntax for set literals ({1,2,3} is a mutable set).
• Dictionary and set comprehensions ({i:i*2foriinrange(3)}).
• Multiple context managers in a singlewith statement.
• A new version of theio library, rewritten in C for performance.
• The ordered-dictionary type described inPEP 372: Adding an Ordered Dictionary to collections.
• The new"," format specifier described inPEP 378: Format Specifier for Thousands Separator.
• Thememoryview object.
• A small subset of theimportlib module,described below.
• Therepr() of a floatx is shorter in many cases: it’s nowbased on the shortest decimal string that’s guaranteed to round backtox. As in previous versions of Python, it’s guaranteed thatfloat(repr(x)) recoversx.
• Float-to-string and string-to-float conversions are correctly rounded.Theround() function is also now correctly rounded.
• ThePyCapsule type, used to provide a C API for extension modules.
• ThePyLong_AsLongAndOverflow() C API function.

Other new Python3-mode warnings include:

• operator.isCallable() andoperator.sequenceIncludes(),which are not supported in 3.x, now trigger warnings.
• The-3 switch now automaticallyenables the-Qwarn switch that causes warningsabout using classic division with integers and long integers.

PEP 372: Adding an Ordered Dictionary to collections¶

Regular Python dictionaries iterate over key/value pairs in arbitrary order.Over the years, a number of authors have written alternative implementationsthat remember the order that the keys were originally inserted. Based onthe experiences from those implementations, 2.7 introduces a newOrderedDict class in thecollections module.

TheOrderedDict API provides the same interface as regulardictionaries but iterates over keys and values in a guaranteed orderdepending on when a key was first inserted:

>>>fromcollectionsimportOrderedDict>>>d=OrderedDict([('first',1),...('second',2),...('third',3)])>>>d.items()[('first', 1), ('second', 2), ('third', 3)]

If a new entry overwrites an existing entry, the original insertionposition is left unchanged:

>>>d['second']=4>>>d.items()[('first', 1), ('second', 4), ('third', 3)]

Deleting an entry and reinserting it will move it to the end:

>>>deld['second']>>>d['second']=5>>>d.items()[('first', 1), ('third', 3), ('second', 5)]

Thepopitem() method has an optionallastargument that defaults to True. Iflast is True, the most recentlyadded key is returned and removed; if it’s False, theoldest key is selected:

>>>od=OrderedDict([(x,0)forxinrange(20)])>>>od.popitem()(19, 0)>>>od.popitem()(18, 0)>>>od.popitem(last=False)(0, 0)>>>od.popitem(last=False)(1, 0)

Comparing two ordered dictionaries checks both the keys and values,and requires that the insertion order was the same:

>>>od1=OrderedDict([('first',1),...('second',2),...('third',3)])>>>od2=OrderedDict([('third',3),...('first',1),...('second',2)])>>>od1==od2False>>># Move 'third' key to the end>>>delod2['third'];od2['third']=3>>>od1==od2True

Comparing anOrderedDict with a regular dictionaryignores the insertion order and just compares the keys and values.

How does theOrderedDict work? It maintains adoubly-linked list of keys, appending new keys to the list as they’re inserted.A secondary dictionary maps keys to their corresponding list node, sodeletion doesn’t have to traverse the entire linked list and thereforeremains O(1).

The standard library now supports use of ordered dictionaries in severalmodules.

• TheConfigParser module uses them by default, meaning thatconfiguration files can now be read, modified, and then written backin their original order.
• The_asdict() method forcollections.namedtuple() now returns an ordered dictionary with thevalues appearing in the same order as the underlying tuple indices.
• Thejson module’sJSONDecoder classconstructor was extended with anobject_pairs_hook parameter toallowOrderedDict instances to be built by the decoder.Support was also added for third-party tools likePyYAML.

PEP 378: Format Specifier for Thousands Separator¶

To make program output more readable, it can be useful to addseparators to large numbers, rendering them as18,446,744,073,709,551,616 instead of 18446744073709551616.

The fully general solution for doing this is thelocale module,which can use different separators (”,” in North America, ”.” inEurope) and different grouping sizes, butlocale is complicatedto use and unsuitable for multi-threaded applications where differentthreads are producing output for different locales.

Therefore, a simple comma-grouping mechanism has been added to themini-language used by thestr.format() method. Whenformatting a floating-point number, simply include a comma between thewidth and the precision:

>>>'{:20,.2f}'.format(18446744073709551616.0)'18,446,744,073,709,551,616.00'

When formatting an integer, include the comma after the width:

>>>'{:20,d}'.format(18446744073709551616)'18,446,744,073,709,551,616'

This mechanism is not adaptable at all; commas are always used as theseparator and the grouping is always into three-digit groups. Thecomma-formatting mechanism isn’t as general as thelocalemodule, but it’s easier to use.

PEP 389: The argparse Module for Parsing Command Lines¶

Theargparse module for parsing command-line arguments wasadded as a more powerful replacement for theoptparse module.

This means Python now supports three different modules for parsingcommand-line arguments:getopt,optparse, andargparse. Thegetopt module closely resembles the Clibrary’sgetopt() function, so it remains useful if you’re writing aPython prototype that will eventually be rewritten in C.optparse becomes redundant, but there are no plans to remove itbecause there are many scripts still using it, and there’s noautomated way to update these scripts. (Making theargparseAPI consistent withoptparse‘s interface was discussed butrejected as too messy and difficult.)

In short, if you’re writing a new script and don’t need to worryabout compatibility with earlier versions of Python, useargparse instead ofoptparse.

Here’s an example:

importargparseparser=argparse.ArgumentParser(description='Command-line example.')# Add optional switchesparser.add_argument('-v',action='store_true',dest='is_verbose',help='produce verbose output')parser.add_argument('-o',action='store',dest='output',metavar='FILE',help='direct output to FILE instead of stdout')parser.add_argument('-C',action='store',type=int,dest='context',metavar='NUM',default=0,help='display NUM lines of added context')# Allow any number of additional arguments.parser.add_argument(nargs='*',action='store',dest='inputs',help='input filenames (default is stdin)')args=parser.parse_args()printargs.__dict__

Unless you override it,-h and--help switchesare automatically added, and produce neatly formatted output:

->./python.exeargparse-example.py--helpusage:argparse-example.py[-h][-v][-oFILE][-CNUM][inputs[inputs...]]Command-lineexample.positionalarguments:inputsinputfilenames(defaultisstdin)optionalarguments:-h,--helpshowthishelpmessageandexit-vproduceverboseoutput-oFILEdirectoutputtoFILEinsteadofstdout-CNUMdisplayNUMlinesofaddedcontext

As withoptparse, the command-line switches and argumentsare returned as an object with attributes named by thedest parameters:

->./python.exeargparse-example.py-v{'output':None,'is_verbose':True,'context':0,'inputs':[]}->./python.exeargparse-example.py-v-o/tmp/output-C4file1file2{'output':'/tmp/output','is_verbose':True,'context':4,'inputs':['file1','file2']}

argparse has much fancier validation thanoptparse; youcan specify an exact number of arguments as an integer, 0 or morearguments by passing'*', 1 or more by passing'+', or anoptional argument with'?'. A top-level parser can containsub-parsers to define subcommands that have different sets ofswitches, as insvncommit,svncheckout, etc. You canspecify an argument’s type asFileType, which willautomatically open files for you and understands that'-' meansstandard input or output.

PEP 391: Dictionary-Based Configuration For Logging¶

Thelogging module is very flexible; applications can definea tree of logging subsystems, and each logger in this tree can filterout certain messages, format them differently, and direct messages toa varying number of handlers.

All this flexibility can require a lot of configuration. You canwrite Python statements to create objects and set their properties,but a complex set-up requires verbose but boring code.logging also supports afileConfig()function that parses a file, but the file format doesn’t supportconfiguring filters, and it’s messier to generate programmatically.

Python 2.7 adds adictConfig() function thatuses a dictionary to configure logging. There are many ways toproduce a dictionary from different sources: construct one with code;parse a file containing JSON; or use a YAML parsing library if one isinstalled.

The following example configures two loggers, the root logger and alogger named “network”. Messages sent to the root logger will besent to the system log using the syslog protocol, and messagesto the “network” logger will be written to anetwork.log filethat will be rotated once the log reaches 1Mb.

importloggingimportlogging.configconfigdict={'version':1,# Configuration schema in use; must be 1 for now'formatters':{'standard':{'format':('%(asctime)s %(name)-15s ''%(levelname)-8s %(message)s')}},'handlers':{'netlog':{'backupCount':10,'class':'logging.handlers.RotatingFileHandler','filename':'/logs/network.log','formatter':'standard','level':'INFO','maxBytes':1024*1024},'syslog':{'class':'logging.handlers.SysLogHandler','formatter':'standard','level':'ERROR'}},# Specify all the subordinate loggers'loggers':{'network':{'handlers':['netlog']}},# Specify properties of the root logger'root':{'handlers':['syslog']},}# Set up configurationlogging.config.dictConfig(configdict)# As an example, log two error messageslogger=logging.getLogger('/')logger.error('Database not found')netlogger=logging.getLogger('network')netlogger.error('Connection failed')

Three smaller enhancements to thelogging module, allimplemented by Vinay Sajip, are:

• TheSysLogHandler class now supportssyslogging over TCP. The constructor has asocktype parametergiving the type of socket to use, eithersocket.SOCK_DGRAMfor UDP orsocket.SOCK_STREAM for TCP. The defaultprotocol remains UDP.
• Logger instances gained agetChild() method that retrieves adescendant logger using a relative path. For example,once you retrieve a logger by doinglog=getLogger('app'),callinglog.getChild('network.listen') is equivalent togetLogger('app.network.listen').
• TheLoggerAdapter class gained aisEnabledFor() methodthat takes alevel and returns whether the underlying logger wouldprocess a message of that level of importance.

PEP 3106: Dictionary Views¶

The dictionary methodskeys(),values(), anditems()are different in Python 3.x. They return an object called aviewinstead of a fully materialized list.

It’s not possible to change the return values ofkeys(),values(), anditems() in Python 2.7 because too much codewould break. Instead the 3.x versions were added under the new namesviewkeys(),viewvalues(), andviewitems().

>>>d=dict((i*10,chr(65+i))foriinrange(26))>>>d{0: 'A', 130: 'N', 10: 'B', 140: 'O', 20: ..., 250: 'Z'}>>>d.viewkeys()dict_keys([0, 130, 10, 140, 20, 150, 30, ..., 250])

Views can be iterated over, but the key and item views also behavelike sets. The& operator performs intersection, and|performs a union:

>>>d1=dict((i*10,chr(65+i))foriinrange(26))>>>d2=dict((i**.5,i)foriinrange(1000))>>>d1.viewkeys()&d2.viewkeys()set([0.0, 10.0, 20.0, 30.0])>>>d1.viewkeys()|range(0,30)set([0, 1, 130, 3, 4, 5, 6, ..., 120, 250])

The view keeps track of the dictionary and its contents change as thedictionary is modified:

>>>vk=d.viewkeys()>>>vkdict_keys([0, 130, 10, ..., 250])>>>d[260]='&'>>>vkdict_keys([0, 130, 260, 10, ..., 250])

However, note that you can’t add or remove keys while you’re iteratingover the view:

>>>forkinvk:...d[k*2]=k...Traceback (most recent call last):  File"<stdin>", line1, in<module>RuntimeError:dictionary changed size during iteration

You can use the view methods in Python 2.x code, and the 2to3converter will change them to the standardkeys(),values(), anditems() methods.

PEP 3137: The memoryview Object¶

Thememoryview object provides a view of another object’smemory content that matches thebytes type’s interface.

>>>importstring>>>m=memoryview(string.letters)>>>m<memory at 0x37f850>>>>len(m)# Returns length of underlying object52>>>m[0],m[25],m[26]# Indexing returns one byte('a', 'z', 'A')>>>m2=m[0:26]# Slicing returns another memoryview>>>m2<memory at 0x37f080>

The content of the view can be converted to a string of bytes ora list of integers:

>>>m2.tobytes()'abcdefghijklmnopqrstuvwxyz'>>>m2.tolist()[97, 98, 99, 100, 101, 102, 103, ... 121, 122]>>>

memoryview objects allow modifying the underlying object ifit’s a mutable object.

>>>m2[0]=75Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:cannot modify read-only memory>>>b=bytearray(string.letters)# Creating a mutable object>>>bbytearray(b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ')>>>mb=memoryview(b)>>>mb[0]='*'# Assign to view, changing the bytearray.>>>b[0:5]# The bytearray has been changed.bytearray(b'*bcde')>>>

Other Language Changes¶

Some smaller changes made to the core Python language are:

• The syntax for set literals has been backported from Python 3.x.Curly brackets are used to surround the contents of the resultingmutable set; set literals aredistinguished from dictionaries by not containing colons and values.{} continues to represent an empty dictionary; useset() for an empty set.

  >>>{1,2,3,4,5}set([1, 2, 3, 4, 5])>>>set()# empty setset([])>>>{}# empty dict{}

  Backported by Alexandre Vassalotti;issue 2335.

• Dictionary and set comprehensions are another feature backported from3.x, generalizing list/generator comprehensions to usethe literal syntax for sets and dictionaries.

  >>>{x:x*xforxinrange(6)}{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}>>>{('a'*x)forxinrange(6)}set(['', 'a', 'aa', 'aaa', 'aaaa', 'aaaaa'])

  Backported by Alexandre Vassalotti;issue 2333.

• Thewith statement can now use multiple context managersin one statement. Context managers are processed from left to rightand each one is treated as beginning a newwith statement.This means that:

  withA()asa,B()asb:...suiteofstatements...

  is equivalent to:

  withA()asa:withB()asb:...suiteofstatements...

  Thecontextlib.nested() function provides a very similarfunction, so it’s no longer necessary and has been deprecated.

  (Proposed inhttp://codereview.appspot.com/53094; implemented byGeorg Brandl.)

• Conversions between floating-point numbers and strings arenow correctly rounded on most platforms. These conversions occurin many different places:str() onfloats and complex numbers; thefloat andcomplexconstructors;numeric formatting; serializing anddeserializing floats and complex numbers using themarshal,pickleandjson modules;parsing of float and imaginary literals in Python code;andDecimal-to-float conversion.

  Related to this, therepr() of a floating-point numberxnow returns a result based on the shortest decimal string that’sguaranteed to round back tox under correct rounding (withround-half-to-even rounding mode). Previously it gave a stringbased on rounding x to 17 decimal digits.

  The rounding library responsible for this improvement works onWindows and on Unix platforms using the gcc, icc, or suncccompilers. There may be a small number of platforms where correctoperation of this code cannot be guaranteed, so the code is notused on such systems. You can find out which code is being usedby checkingsys.float_repr_style, which will beshortif the new code is in use andlegacy if it isn’t.

  Implemented by Eric Smith and Mark Dickinson, using David Gay’sdtoa.c library;issue 7117.

• Conversions from long integers and regular integers to floatingpoint now round differently, returning the floating-point numberclosest to the number. This doesn’t matter for small integers thatcan be converted exactly, but for large numbers that willunavoidably lose precision, Python 2.7 now approximates moreclosely. For example, Python 2.6 computed the following:

  >>>n=295147905179352891391>>>float(n)2.9514790517935283e+20>>>n-long(float(n))65535L

  Python 2.7’s floating-point result is larger, but much closer to thetrue value:

  >>>n=295147905179352891391>>>float(n)2.9514790517935289e+20>>>n-long(float(n))-1L

  (Implemented by Mark Dickinson;issue 3166.)

  Integer division is also more accurate in its rounding behaviours. (Alsoimplemented by Mark Dickinson;issue 1811.)

• Implicit coercion for complex numbers has been removed; the interpreterwill no longer ever attempt to call a__coerce__() method on complexobjects. (Removed by Meador Inge and Mark Dickinson;issue 5211.)

• Thestr.format() method now supports automatic numbering of the replacementfields. This makes usingstr.format() more closely resemble using%s formatting:

  >>>'{}:{}:{}'.format(2009,04,'Sunday')'2009:4:Sunday'>>>'{}:{}:{day}'.format(2009,4,day='Sunday')'2009:4:Sunday'

  The auto-numbering takes the fields from left to right, so the first{...}specifier will use the first argument tostr.format(), the nextspecifier will use the next argument, and so on. You can’t mix auto-numberingand explicit numbering – either number all of your specifier fields or noneof them – but you can mix auto-numbering and named fields, as in the secondexample above. (Contributed by Eric Smith;issue 5237.)

  Complex numbers now correctly support usage withformat(),and default to being right-aligned.Specifying a precision or comma-separation applies to both the realand imaginary parts of the number, but a specified field width andalignment is applied to the whole of the resulting1.5+3joutput. (Contributed by Eric Smith;issue 1588 andissue 7988.)

  The ‘F’ format code now always formats its output using uppercase characters,so it will now produce ‘INF’ and ‘NAN’.(Contributed by Eric Smith;issue 3382.)

  A low-level change: theobject.__format__() method now triggersaPendingDeprecationWarning if it’s passed a format string,because the__format__() method forobject convertsthe object to a string representation and formats that. Previouslythe method silently applied the format string to the stringrepresentation, but that could hide mistakes in Python code. Ifyou’re supplying formatting information such as an alignment orprecision, presumably you’re expecting the formatting to be appliedin some object-specific way. (Fixed by Eric Smith;issue 7994.)

• Theint() andlong() types gained abit_lengthmethod that returns the number of bits necessary to representits argument in binary:

  >>>n=37>>>bin(n)'0b100101'>>>n.bit_length()6>>>n=2**123-1>>>n.bit_length()123>>>(n+1).bit_length()124

  (Contributed by Fredrik Johansson and Victor Stinner;issue 3439.)

• Theimport statement will no longer try an absolute importif a relative import (e.g.from.osimportsep) fails. Thisfixes a bug, but could possibly break certainimportstatements that were only working by accident. (Fixed by Meador Inge;issue 7902.)

• It’s now possible for a subclass of the built-inunicode typeto override the__unicode__() method. (Implemented byVictor Stinner;issue 1583863.)

• Thebytearray type’stranslate() method now acceptsNone as its first argument. (Fixed by Georg Brandl;issue 4759.)

• When using@classmethod and@staticmethod to wrapmethods as class or static methods, the wrapper object nowexposes the wrapped function as their__func__ attribute.(Contributed by Amaury Forgeot d’Arc, after a suggestion byGeorge Sakkis;issue 5982.)

• When a restricted set of attributes were set using__slots__,deleting an unset attribute would not raiseAttributeErroras you would expect. Fixed by Benjamin Peterson;issue 7604.)

• Two new encodings are now supported: “cp720”, used primarily forArabic text; and “cp858”, a variant of CP 850 that adds the eurosymbol. (CP720 contributed by Alexander Belchenko and AmauryForgeot d’Arc inissue 1616979; CP858 contributed by Tim Hatch inissue 8016.)

• Thefile object will now set thefilename attributeon theIOError exception when trying to open a directoryon POSIX platforms (noted by Jan Kaliszewski;issue 4764), andnow explicitly checks for and forbids writing to read-only file objectsinstead of trusting the C library to catch and report the error(fixed by Stefan Krah;issue 5677).

• The Python tokenizer now translates line endings itself, so thecompile() built-in function now accepts code using anyline-ending convention. Additionally, it no longer requires that thecode end in a newline.

• Extra parentheses in function definitions are illegal in Python 3.x,meaning that you get a syntax error fromdeff((x)):pass. InPython3-warning mode, Python 2.7 will now warn about this odd usage.(Noted by James Lingard;issue 7362.)

• It’s now possible to create weak references to old-style classobjects. New-style classes were always weak-referenceable. (Fixedby Antoine Pitrou;issue 8268.)

• When a module object is garbage-collected, the module’s dictionary isnow only cleared if no one else is holding a reference to thedictionary (issue 7140).

exportPYTHONWARNINGS=all,error:::Cookie:0

New and Improved Modules¶

As in every release, Python’s standard library received a number ofenhancements and bug fixes. Here’s a partial list of the most notablechanges, sorted alphabetically by module name. Consult theMisc/NEWS file in the source tree for a more complete list ofchanges, or look through the Subversion logs for all the details.

• Thebdb module’s base debugging classBdbgained a feature for skipping modules. The constructornow takes an iterable containing glob-style patterns such asdjango.*; the debugger will not step into stack framesfrom a module that matches one of these patterns.(Contributed by Maru Newby after a suggestion bySenthil Kumaran;issue 5142.)

• Thebinascii module now supports the buffer API, so it can beused withmemoryview instances and other similar buffer objects.(Backported from 3.x by Florent Xicluna;issue 7703.)

• Updated module: thebsddb module has been updated from 4.7.2devel9to version 4.8.4 ofthe pybsddb package.The new version features better Python 3.x compatibility, various bug fixes,and adds several new BerkeleyDB flags and methods.(Updated by Jesús Cea Avión;issue 8156. The pybsddbchangelog can be read athttp://hg.jcea.es/pybsddb/file/tip/ChangeLog.)

• Thebz2 module’sBZ2File now supports the contextmanagement protocol, so you can writewithbz2.BZ2File(...)asf:.(Contributed by Hagen Fürstenau;issue 3860.)

• New class: theCounter class in thecollectionsmodule is useful for tallying data.Counter instancesbehave mostly like dictionaries but return zero for missing keys instead ofraising aKeyError:

  >>>fromcollectionsimportCounter>>>c=Counter()>>>forletterin'here is a sample of english text':...c[letter]+=1...>>>cCounter({' ': 6, 'e': 5, 's': 3, 'a': 2, 'i': 2, 'h': 2,'l': 2, 't': 2, 'g': 1, 'f': 1, 'm': 1, 'o': 1, 'n': 1,'p': 1, 'r': 1, 'x': 1})>>>c['e']5>>>c['z']0

  There are three additionalCounter methods.most_common() returns the N most commonelements and their counts.elements()returns an iterator over the contained elements, repeating eachelement as many times as its count.subtract() takes an iterable andsubtracts one for each element instead of adding; if the argument isa dictionary or anotherCounter, the counts aresubtracted.

  >>>c.most_common(5)[(' ', 6), ('e', 5), ('s', 3), ('a', 2), ('i', 2)]>>>c.elements()->   'a', 'a', ' ', ' ', ' ', ' ', ' ', ' ',   'e', 'e', 'e', 'e', 'e', 'g', 'f', 'i', 'i',   'h', 'h', 'm', 'l', 'l', 'o', 'n', 'p', 's',   's', 's', 'r', 't', 't', 'x'>>>c['e']5>>>c.subtract('very heavy on the letter e')>>>c['e']# Count is now lower-1

  Contributed by Raymond Hettinger;issue 1696199.

  New class:OrderedDict is described in the earliersectionPEP 372: Adding an Ordered Dictionary to collections.

  New method: Thedeque data type now has acount() method that returns the number ofcontained elements equal to the supplied argumentx, and areverse() method that reverses the elementsof the deque in-place.deque also exposes its maximumlength as the read-onlymaxlen attribute.(Both features added by Raymond Hettinger.)

  Thenamedtuple class now has an optionalrename parameter.Ifrename is true, field names that are invalid because they’vebeen repeated or aren’t legal Python identifiers will berenamed to legal names that are derived from the field’sposition within the list of fields:

  >>>fromcollectionsimportnamedtuple>>>T=namedtuple('T',['field1','$illegal','for','field2'],rename=True)>>>T._fields('field1', '_1', '_2', 'field2')

  (Added by Raymond Hettinger;issue 1818.)

  Finally, theMapping abstract base class nowreturnsNotImplemented if a mapping is compared toanother type that isn’t aMapping.(Fixed by Daniel Stutzbach;issue 8729.)

• Constructors for the parsing classes in theConfigParser module nowtake aallow_no_value parameter, defaulting to false; if true,options without values will be allowed. For example:

  >>>importConfigParser,StringIO>>>sample_config="""...[mysqld]...user = mysql...pid-file = /var/run/mysqld/mysqld.pid...skip-bdb...""">>>config=ConfigParser.RawConfigParser(allow_no_value=True)>>>config.readfp(StringIO.StringIO(sample_config))>>>config.get('mysqld','user')'mysql'>>>printconfig.get('mysqld','skip-bdb')None>>>printconfig.get('mysqld','unknown')Traceback (most recent call last):...NoOptionError:No option 'unknown' in section: 'mysqld'

  (Contributed by Mats Kindahl;issue 7005.)

• Deprecated function:contextlib.nested(), which allowshandling more than one context manager with a singlewithstatement, has been deprecated, because thewith statementnow supports multiple context managers.

• Thecookielib module now ignores cookies that have an invalidversion field, one that doesn’t contain an integer value. (Fixed byJohn J. Lee;issue 3924.)

• Thecopy module’sdeepcopy() function will nowcorrectly copy bound instance methods. (Implemented byRobert Collins;issue 1515.)

• Thectypes module now always convertsNone to a C NULLpointer for arguments declared as pointers. (Changed by ThomasHeller;issue 4606.) The underlyinglibffi library has been updated to version3.0.9, containing various fixes for different platforms. (Updatedby Matthias Klose;issue 8142.)

• New method: thedatetime module’stimedelta classgained atotal_seconds() method that returns thenumber of seconds in the duration. (Contributed by Brian Quinlan;issue 5788.)

• New method: theDecimal class gained afrom_float() class method that performs an exactconversion of a floating-point number to aDecimal.This exact conversion strives for theclosest decimal approximation to the floating-point representation’s value;the resulting decimal value will therefore still include the inaccuracy,if any.For example,Decimal.from_float(0.1) returnsDecimal('0.1000000000000000055511151231257827021181583404541015625').(Implemented by Raymond Hettinger;issue 4796.)

  Comparing instances ofDecimal with floating-pointnumbers now produces sensible results based on the numeric valuesof the operands. Previously such comparisons would fall back toPython’s default rules for comparing objects, which produced arbitraryresults based on their type. Note that you still cannot combineDecimal and floating-point in other operations such as addition,since you should be explicitly choosing how to convert between float andDecimal.(Fixed by Mark Dickinson;issue 2531.)

  The constructor forDecimal now acceptsfloating-point numbers (added by Raymond Hettinger;issue 8257)and non-European Unicode characters such as Arabic-Indic digits(contributed by Mark Dickinson;issue 6595).

  Most of the methods of theContext class now accept integersas well asDecimal instances; the only exceptions are thecanonical() andis_canonical()methods. (Patch by Juan José Conti;issue 7633.)

  When usingDecimal instances with a string’sformat() method, the default alignment was previouslyleft-alignment. This has been changed to right-alignment, which ismore sensible for numeric types. (Changed by Mark Dickinson;issue 6857.)

  Comparisons involving a signaling NaN value (orsNAN) now signalInvalidOperation instead of silently returning a true orfalse value depending on the comparison operator. Quiet NaN values(orNaN) are now hashable. (Fixed by Mark Dickinson;issue 7279.)

• Thedifflib module now produces output that is morecompatible with moderndiff/patch toolsthrough one small change, using a tab character instead of spaces asa separator in the header giving the filename. (Fixed by AnatolyTechtonik;issue 7585.)

• The Distutilssdist command now always regenerates theMANIFEST file, since even if theMANIFEST.in orsetup.py files haven’t been modified, the user might havecreated some new files that should be included.(Fixed by Tarek Ziadé;issue 8688.)

• Thedoctest module’sIGNORE_EXCEPTION_DETAIL flagwill now ignore the name of the module containing the exceptionbeing tested. (Patch by Lennart Regebro;issue 7490.)

• Theemail module’sMessage class willnow accept a Unicode-valued payload, automatically converting thepayload to the encoding specified byoutput_charset.(Added by R. David Murray;issue 1368247.)

• TheFraction class now accepts a single float orDecimal instance, or two rational numbers, asarguments to its constructor. (Implemented by Mark Dickinson;rationals added inissue 5812, and float/decimal inissue 8294.)

  Ordering comparisons (<,<=,>,>=) betweenfractions and complex numbers now raise aTypeError.This fixes an oversight, making theFraction match the othernumeric types.

• New class:FTP_TLS intheftplib module provides secure FTPconnections using TLS encapsulation of authentication as well assubsequent control and data transfers.(Contributed by Giampaolo Rodola;issue 2054.)

  Thestorbinary() method for binary uploads can now restartuploads thanks to an addedrest parameter (patch by Pablo Mouzo;issue 6845.)

• New class decorator:total_ordering() in thefunctoolsmodule takes a class that defines an__eq__() method and one of__lt__(),__le__(),__gt__(), or__ge__(),and generates the missing comparison methods. Since the__cmp__() method is being deprecated in Python 3.x,this decorator makes it easier to define ordered classes.(Added by Raymond Hettinger;issue 5479.)

  New function:cmp_to_key() will take an old-style comparisonfunction that expects two arguments and return a new callable thatcan be used as thekey parameter to functions such assorted(),min() andmax(), etc. The primaryintended use is to help with making code compatible with Python 3.x.(Added by Raymond Hettinger.)

• New function: thegc module’sis_tracked() returnstrue if a given instance is tracked by the garbage collector, falseotherwise. (Contributed by Antoine Pitrou;issue 4688.)

• Thegzip module’sGzipFile now supports the contextmanagement protocol, so you can writewithgzip.GzipFile(...)asf:(contributed by Hagen Fürstenau;issue 3860), and it now implementstheio.BufferedIOBase ABC, so you can wrap it withio.BufferedReader for faster processing(contributed by Nir Aides;issue 7471).It’s also now possible to override the modification timerecorded in a gzipped file by providing an optional timestamp tothe constructor. (Contributed by Jacques Frechet;issue 4272.)

  Files in gzip format can be padded with trailing zero bytes; thegzip module will now consume these trailing bytes. (Fixed byTadek Pietraszek and Brian Curtin;issue 2846.)

• New attribute: thehashlib module now has analgorithmsattribute containing a tuple naming the supported algorithms.In Python 2.7,hashlib.algorithms contains('md5','sha1','sha224','sha256','sha384','sha512').(Contributed by Carl Chenet;issue 7418.)

• The defaultHTTPResponse class used by thehttplib module nowsupports buffering, resulting in much faster reading of HTTP responses.(Contributed by Kristján Valur Jónsson;issue 4879.)

  TheHTTPConnection andHTTPSConnection classesnow support asource_address parameter, a(host,port) 2-tuplegiving the source address that will be used for the connection.(Contributed by Eldon Ziegler;issue 3972.)

• Theihooks module now supports relative imports. Note thatihooks is an older module for customizing imports,superseded by theimputil module added in Python 2.0.(Relative import support added by Neil Schemenauer.)

• Theimaplib module now supports IPv6 addresses.(Contributed by Derek Morr;issue 1655.)

• New function: theinspect module’sgetcallargs()takes a callable and its positional and keyword arguments,and figures out which of the callable’s parameters will receive each argument,returning a dictionary mapping argument names to their values. For example:

  >>>frominspectimportgetcallargs>>>deff(a,b=1,*pos,**named):...pass>>>getcallargs(f,1,2,3){'a': 1, 'b': 2, 'pos': (3,), 'named': {}}>>>getcallargs(f,a=2,x=4){'a': 2, 'b': 1, 'pos': (), 'named': {'x': 4}}>>>getcallargs(f)Traceback (most recent call last):...TypeError:f() takes at least 1 argument (0 given)

  Contributed by George Sakkis;issue 3135.

• Updated module: Theio library has been upgraded to the version shipped withPython 3.1. For 3.1, the I/O library was entirely rewritten in Cand is 2 to 20 times faster depending on the task being performed. Theoriginal Python version was renamed to the_pyio module.

  One minor resulting change: theio.TextIOBase class nowhas anerrors attribute giving the error settingused for encoding and decoding errors (one of'strict','replace','ignore').

  Theio.FileIO class now raises anOSError when passedan invalid file descriptor. (Implemented by Benjamin Peterson;issue 4991.) Thetruncate() method now preserves thefile position; previously it would change the file position to theend of the new file. (Fixed by Pascal Chambon;issue 6939.)

• New function:itertools.compress(data,selectors) takes twoiterators. Elements ofdata are returned if the correspondingvalue inselectors is true:

  itertools.compress('ABCDEF',[1,0,1,0,1,1])=>A,C,E,F

  New function:itertools.combinations_with_replacement(iter,r)returns all the possibler-length combinations of elements from theiterableiter. Unlikecombinations(), individual elementscan be repeated in the generated combinations:

  itertools.combinations_with_replacement('abc',2)=>('a','a'),('a','b'),('a','c'),('b','b'),('b','c'),('c','c')

  Note that elements are treated as unique depending on their positionin the input, not their actual values.

  Theitertools.count() function now has astep argument thatallows incrementing by values other than 1.count() alsonow allows keyword arguments, and using non-integer values such asfloats orDecimal instances. (Implemented by RaymondHettinger;issue 5032.)

  itertools.combinations() anditertools.product()previously raisedValueError for values ofr larger thanthe input iterable. This was deemed a specification error, so theynow return an empty iterator. (Fixed by Raymond Hettinger;issue 4816.)

• Updated module: Thejson module was upgraded to version 2.0.9 of thesimplejson package, which includes a C extension that makesencoding and decoding faster.(Contributed by Bob Ippolito;issue 4136.)

  To support the newcollections.OrderedDict type,json.load()now has an optionalobject_pairs_hook parameter that will be calledwith any object literal that decodes to a list of pairs.(Contributed by Raymond Hettinger;issue 5381.)

• Themailbox module’sMaildir class now records thetimestamp on the directories it reads, and only re-reads them if themodification time has subsequently changed. This improvesperformance by avoiding unneeded directory scans. (Fixed byA.M. Kuchling and Antoine Pitrou;issue 1607951,issue 6896.)

• New functions: themath module gainederf() anderfc() for the error function and the complementary error function,expm1() which computese**x-1 with more precision thanusingexp() and subtracting 1,gamma() for the Gamma function, andlgamma() for the natural log of the Gamma function.(Contributed by Mark Dickinson and nirinA raseliarison;issue 3366.)

• Themultiprocessing module’sManager* classescan now be passed a callable that will be called whenevera subprocess is started, along with a set of arguments that will bepassed to the callable.(Contributed by lekma;issue 5585.)

  ThePool class, which controls a pool of worker processes,now has an optionalmaxtasksperchild parameter. Worker processeswill perform the specified number of tasks and then exit, causing thePool to start a new worker. This is useful if tasks may leakmemory or other resources, or if some tasks will cause the worker tobecome very large.(Contributed by Charles Cazabon;issue 6963.)

• Thenntplib module now supports IPv6 addresses.(Contributed by Derek Morr;issue 1664.)

• New functions: theos module wraps the following POSIX systemcalls:getresgid() andgetresuid(), which return thereal, effective, and saved GIDs and UIDs;setresgid() andsetresuid(), which setreal, effective, and saved GIDs and UIDs to new values;initgroups(), which initialize the group access listfor the current process. (GID/UID functionscontributed by Travis H.;issue 6508. Support for initgroups addedby Jean-Paul Calderone;issue 7333.)

  Theos.fork() function now re-initializes the import lock inthe child process; this fixes problems on Solaris whenfork()is called from a thread. (Fixed by Zsolt Cserna;issue 7242.)

• In theos.path module, thenormpath() andabspath() functions now preserve Unicode; if their input pathis a Unicode string, the return value is also a Unicode string.(normpath() fixed by Matt Giuca inissue 5827;abspath() fixed by Ezio Melotti inissue 3426.)

• Thepydoc module now has help for the various symbols that Pythonuses. You can now dohelp('<<') orhelp('@'), for example.(Contributed by David Laban;issue 4739.)

• There module’ssplit(),sub(), andsubn()now accept an optionalflags argument, for consistency with theother functions in the module. (Added by Gregory P. Smith.)

• New function:run_path() in therunpy modulewill execute the code at a providedpath argument.path can bethe path of a Python source file (example.py), a compiledbytecode file (example.pyc), a directory(./package/), or a zip archive (example.zip). If adirectory or zip path is provided, it will be added to the front ofsys.path and the module__main__ will be imported. It’sexpected that the directory or zip contains a__main__.py;if it doesn’t, some other__main__.py might be imported froma location later insys.path. This makes more of the machineryofrunpy available to scripts that want to mimic the wayPython’s command line processes an explicit path name.(Added by Nick Coghlan;issue 6816.)

• New function: in theshutil module,make_archive()takes a filename, archive type (zip or tar-format), and a directorypath, and creates an archive containing the directory’s contents.(Added by Tarek Ziadé.)

  shutil‘scopyfile() andcopytree()functions now raise aSpecialFileError exception whenasked to copy a named pipe. Previously the code would treatnamed pipes like a regular file by opening them for reading, andthis would block indefinitely. (Fixed by Antoine Pitrou;issue 3002.)

• Thesignal module no longer re-installs the signal handlerunless this is truly necessary, which fixes a bug that could make itimpossible to catch the EINTR signal robustly. (Fixed byCharles-François Natali;issue 8354.)

• New functions: in thesite module, three new functionsreturn various site- and user-specific paths.getsitepackages() returns a list containing allglobal site-packages directories,getusersitepackages() returns the path of the user’ssite-packages directory, andgetuserbase() returns the value of theUSER_BASEenvironment variable, giving the path to a directory that can be usedto store data.(Contributed by Tarek Ziadé;issue 6693.)

  Thesite module now reports exceptions occurringwhen thesitecustomize module is imported, and will no longercatch and swallow theKeyboardInterrupt exception. (Fixed byVictor Stinner;issue 3137.)

• Thecreate_connection() functiongained asource_address parameter, a(host,port) 2-tuplegiving the source address that will be used for the connection.(Contributed by Eldon Ziegler;issue 3972.)

  Therecv_into() andrecvfrom_into()methods will now write into objects that support the buffer API, most usefullythebytearray andmemoryview objects. (Implemented byAntoine Pitrou;issue 8104.)

• TheSocketServer module’sTCPServer class nowsupports socket timeouts and disabling the Nagle algorithm.Thedisable_nagle_algorithm class attributedefaults to False; if overridden to be True,new request connections will have the TCP_NODELAY option set toprevent buffering many small sends into a single TCP packet.Thetimeout class attribute can holda timeout in seconds that will be applied to the request socket; ifno request is received within that time,handle_timeout()will be called andhandle_request() will return.(Contributed by Kristján Valur Jónsson;issue 6192 andissue 6267.)

• Updated module: thesqlite3 module has been updated toversion 2.6.0 of thepysqlite package. Version 2.6.0 includes a number of bugfixes, and addsthe ability to load SQLite extensions from shared libraries.Call theenable_load_extension(True) method to enable extensions,and then callload_extension() to load a particular shared library.(Updated by Gerhard Häring.)

• Thessl module’sssl.SSLSocket objects now support thebuffer API, which fixed a test suite failure (fix by Antoine Pitrou;issue 7133) and automatically setOpenSSL’sSSL_MODE_AUTO_RETRY, which will prevent an errorcode being returned fromrecv() operations that trigger an SSLrenegotiation (fix by Antoine Pitrou;issue 8222).

  Thessl.wrap_socket() constructor function now takes aciphers argument that’s a string listing the encryption algorithmsto be allowed; the format of the string is describedin the OpenSSL documentation.(Added by Antoine Pitrou;issue 8322.)

  Another change makes the extension load all of OpenSSL’s ciphers anddigest algorithms so that they’re all available. Some SSLcertificates couldn’t be verified, reporting an “unknown algorithm”error. (Reported by Beda Kosata, and fixed by Antoine Pitrou;issue 8484.)

  The version of OpenSSL being used is now available as the moduleattributesssl.OPENSSL_VERSION (a string),ssl.OPENSSL_VERSION_INFO (a 5-tuple), andssl.OPENSSL_VERSION_NUMBER (an integer). (Added by AntoinePitrou;issue 8321.)

• Thestruct module will no longer silently ignore overflowerrors when a value is too large for a particular integer formatcode (one ofbBhHiIlLqQ); it now always raises astruct.error exception. (Changed by Mark Dickinson;issue 1523.) Thepack() function will alsoattempt to use__index__() to convert and pack non-integersbefore trying the__int__() method or reporting an error.(Changed by Mark Dickinson;issue 8300.)

• New function: thesubprocess module’scheck_output() runs a command with a specified set of argumentsand returns the command’s output as a string when the command runs withouterror, or raises aCalledProcessError exception otherwise.

  >>>subprocess.check_output(['df','-h','.'])'Filesystem     Size   Used  Avail Capacity  Mounted on\n/dev/disk0s2    52G    49G   3.0G    94%    /\n'>>>subprocess.check_output(['df','-h','/bogus'])  ...subprocess.CalledProcessError: Command '['df', '-h', '/bogus']' returned non-zero exit status 1

  (Contributed by Gregory P. Smith.)

  Thesubprocess module will now retry its internal system callson receiving anEINTR signal. (Reported by several people; finalpatch by Gregory P. Smith inissue 1068268.)

• New function:is_declared_global() in thesymtable modulereturns true for variables that are explicitly declared to be global,false for ones that are implicitly global.(Contributed by Jeremy Hylton.)

• Thesyslog module will now use the value ofsys.argv[0] as theidentifier instead of the previous default value of'python'.(Changed by Sean Reifschneider;issue 8451.)

• Thesys.version_info value is now a named tuple, with attributesnamedmajor,minor,micro,releaselevel, andserial. (Contributed by RossLight;issue 4285.)

  sys.getwindowsversion() also returns a named tuple,with attributes namedmajor,minor,build,platform,service_pack,service_pack_major,service_pack_minor,suite_mask, andproduct_type. (Contributed by Brian Curtin;issue 7766.)

• Thetarfile module’s default error handling has changed, tono longer suppress fatal errors. The default error level was previously 0,which meant that errors would only result in a message being written to thedebug log, but because the debug log is not activated by default,these errors go unnoticed. The default error level is now 1,which raises an exception if there’s an error.(Changed by Lars Gustäbel;issue 7357.)

  tarfile now supports filtering theTarInfoobjects being added to a tar file. When you calladd(),you may supply an optionalfilter argumentthat’s a callable. Thefilter callable will be passed theTarInfo for every file being added, and can modify and return it.If the callable returnsNone, the file will be excluded from theresulting archive. This is more powerful than the existingexclude argument, which has therefore been deprecated.(Added by Lars Gustäbel;issue 6856.)TheTarFile class also now supports the context manager protocol.(Added by Lars Gustäbel;issue 7232.)

• Thewait() method of thethreading.Event classnow returns the internal flag on exit. This means the method will usuallyreturn true becausewait() is supposed to block until theinternal flag becomes true. The return value will only be false ifa timeout was provided and the operation timed out.(Contributed by Tim Lesher;issue 1674032.)

• The Unicode database provided by theunicodedata module isnow used internally to determine which characters are numeric,whitespace, or represent line breaks. The database alsoincludes information from theUnihan.txt data file (patchby Anders Chrigström and Amaury Forgeot d’Arc;issue 1571184)and has been updated to version 5.2.0 (updated byFlorent Xicluna;issue 8024).

• Theurlparse module’surlsplit() now handlesunknown URL schemes in a fashion compliant withRFC 3986: if theURL is of the form"<something>://...", the text before the:// is treated as the scheme, even if it’s a made-up scheme thatthe module doesn’t know about. This change may break code thatworked around the old behaviour. For example, Python 2.6.4 or 2.5will return the following:

  >>>importurlparse>>>urlparse.urlsplit('invented://host/filename?query')('invented', '', '//host/filename?query', '', '')

  Python 2.7 (and Python 2.6.5) will return:

  >>>importurlparse>>>urlparse.urlsplit('invented://host/filename?query')('invented', 'host', '/filename?query', '', '')

  (Python 2.7 actually produces slightly different output, since itreturns a named tuple instead of a standard tuple.)

  Theurlparse module also supports IPv6 literal addresses as defined byRFC 2732 (contributed by Senthil Kumaran;issue 2987).

  >>>urlparse.urlparse('http://[1080::8:800:200C:417A]/foo')ParseResult(scheme='http', netloc='[1080::8:800:200C:417A]',            path='/foo', params='', query='', fragment='')

• New class: theWeakSet class in theweakrefmodule is a set that only holds weak references to its elements; elementswill be removed once there are no references pointing to them.(Originally implemented in Python 3.x by Raymond Hettinger, and backportedto 2.7 by Michael Foord.)

• The ElementTree library,xml.etree, no longer escapesampersands and angle brackets when outputting an XML processinginstruction (which looks like<?xml-stylesheethref="#style1"?>)or comment (which looks like<!--comment-->).(Patch by Neil Muller;issue 2746.)

• The XML-RPC client and server, provided by thexmlrpclib andSimpleXMLRPCServer modules, have improved performance bysupporting HTTP/1.1 keep-alive and by optionally using gzip encodingto compress the XML being exchanged. The gzip compression iscontrolled by theencode_threshold attribute ofSimpleXMLRPCRequestHandler, which contains a size in bytes;responses larger than this will be compressed.(Contributed by Kristján Valur Jónsson;issue 6267.)

• Thezipfile module’sZipFile now supports the contextmanagement protocol, so you can writewithzipfile.ZipFile(...)asf:.(Contributed by Brian Curtin;issue 5511.)

  zipfile now also supports archiving empty directories andextracts them correctly. (Fixed by Kuba Wieczorek;issue 4710.)Reading files out of an archive is faster, and interleavingread() andreadline() now works correctly.(Contributed by Nir Aides;issue 7610.)

  Theis_zipfile() function nowaccepts a file object, in addition to the path names accepted in earlierversions. (Contributed by Gabriel Genellina;issue 4756.)

  Thewritestr() method now has an optionalcompress_type parameterthat lets you override the default compression method specified in theZipFile constructor. (Contributed by Ronald Oussoren;issue 6003.)

>>>fromimportlibimportimport_module>>>anydbm=import_module('anydbm')# Standard absolute import>>>anydbm<module 'anydbm' from '/p/python/Lib/anydbm.py'>>>># Relative import>>>file_util=import_module('..file_util','distutils.command')>>>file_util<module 'distutils.file_util' from '/python/Lib/distutils/file_util.pyc'>

python-munittestdiscover-stest

withself.assertRaises(KeyError):{}['foo']

Build and C API Changes¶

Changes to Python’s build process and to the C API include:

• The latest release of the GNU Debugger, GDB 7, can bescriptedusing Python.When you begin debugging an executable program P, GDB will look fora file namedP-gdb.py and automatically read it. Dave Malcolmcontributed apython-gdb.py that adds a number ofcommands useful when debugging Python itself. For example,py-up andpy-down go up or down one Python stack frame,which usually corresponds to several C stack frames.py-printprints the value of a Python variable, andpy-bt prints thePython stack trace. (Added as a result ofissue 8032.)

• If you use the.gdbinit file provided with Python,the “pyo” macro in the 2.7 version now works correctly when the thread beingdebugged doesn’t hold the GIL; the macro now acquires it before printing.(Contributed by Victor Stinner;issue 3632.)

• Py_AddPendingCall() is now thread-safe, letting anyworker thread submit notifications to the main Python thread. Thisis particularly useful for asynchronous IO operations.(Contributed by Kristján Valur Jónsson;issue 4293.)

• New function:PyCode_NewEmpty() creates an empty code object;only the filename, function name, and first line number are required.This is useful for extension modules that are attempting toconstruct a more useful traceback stack. Previously suchextensions needed to callPyCode_New(), which had manymore arguments. (Added by Jeffrey Yasskin.)

• New function:PyErr_NewExceptionWithDoc() creates a newexception class, just as the existingPyErr_NewException() does,but takes an extrachar* argument containing the docstring for thenew exception class. (Added by ‘lekma’ on the Python bug tracker;issue 7033.)

• New function:PyFrame_GetLineNumber() takes a frame objectand returns the line number that the frame is currently executing.Previously code would need to get the index of the bytecodeinstruction currently executing, and then look up the line numbercorresponding to that address. (Added by Jeffrey Yasskin.)

• New functions:PyLong_AsLongAndOverflow() andPyLong_AsLongLongAndOverflow() approximates a Python longinteger as a Clong orlonglong.If the number is too large to fit intothe output type, anoverflow flag is set and returned to the caller.(Contributed by Case Van Horsen;issue 7528 andissue 7767.)

• New function: stemming from the rewrite of string-to-float conversion,a newPyOS_string_to_double() function was added. The oldPyOS_ascii_strtod() andPyOS_ascii_atof() functionsare now deprecated.

• New function:PySys_SetArgvEx() sets the value ofsys.argv and can optionally updatesys.path to include thedirectory containing the script named bysys.argv[0] dependingon the value of anupdatepath parameter.

  This function was added to close a security hole for applicationsthat embed Python. The old function,PySys_SetArgv(), wouldalways updatesys.path, and sometimes it would add the currentdirectory. This meant that, if you ran an application embeddingPython in a directory controlled by someone else, attackers couldput a Trojan-horse module in the directory (say, a file namedos.py) that your application would then import and run.

  If you maintain a C/C++ application that embeds Python, checkwhether you’re callingPySys_SetArgv() and carefully considerwhether the application should be usingPySys_SetArgvEx()withupdatepath set to false.

  Security issue reported asCVE-2008-5983;discussed inissue 5753, and fixed by Antoine Pitrou.

• New macros: the Python header files now define the following macros:Py_ISALNUM,Py_ISALPHA,Py_ISDIGIT,Py_ISLOWER,Py_ISSPACE,Py_ISUPPER,Py_ISXDIGIT,andPy_TOLOWER,Py_TOUPPER.All of these functions are analogous to the Cstandard macros for classifying characters, but ignore the currentlocale setting, because inseveral places Python needs to analyze characters in alocale-independent way. (Added by Eric Smith;issue 5793.)

• Removed function:PyEval_CallObject is now only availableas a macro. A function version was being kept around to preserveABI linking compatibility, but that was in 1997; it can certainly bedeleted by now. (Removed by Antoine Pitrou;issue 8276.)

• New format codes: thePyFormat_FromString(),PyFormat_FromStringV(), andPyErr_Format() functions nowaccept%lld and%llu format codes for displayingC’slonglong types.(Contributed by Mark Dickinson;issue 7228.)

• The complicated interaction between threads and process forking hasbeen changed. Previously, the child process created byos.fork() might fail because the child is created with only asingle thread running, the thread performing theos.fork().If other threads were holding a lock, such as Python’s import lock,when the fork was performed, the lock would still be marked as“held” in the new process. But in the child process nothing wouldever release the lock, since the other threads weren’t replicated,and the child process would no longer be able to perform imports.

  Python 2.7 acquires the import lock before performing anos.fork(), and will also clean up any locks created using thethreading module. C extension modules that have internallocks, or that callfork() themselves, will not benefitfrom this clean-up.

  (Fixed by Thomas Wouters;issue 1590864.)

• ThePy_Finalize() function now calls the internalthreading._shutdown() function; this prevents some exceptions frombeing raised when an interpreter shuts down.(Patch by Adam Olsen;issue 1722344.)

• When using thePyMemberDef structure to define attributesof a type, Python will no longer let you try to delete or set aT_STRING_INPLACE attribute.

• Global symbols defined by thectypes module are now prefixedwithPy, or with_ctypes. (Implemented by ThomasHeller;issue 3102.)

• New configure option: the--with-system-expat switch allowsbuilding thepyexpat module to use the system Expat library.(Contributed by Arfrever Frehtes Taifersar Arahesis;issue 7609.)

• New configure option: the--with-valgrind option will now disable the pymallocallocator, which is difficult for the Valgrind memory-error detectorto analyze correctly.Valgrind will therefore be better at detecting memory leaks andoverruns. (Contributed by James Henstridge;issue 2422.)

• New configure option: you can now supply an empty string to--with-dbmliborder= in order to disable all of the variousDBM modules. (Added by Arfrever Frehtes Taifersar Arahesis;issue 6491.)

• Theconfigure script now checks for floating-point rounding bugson certain 32-bit Intel chips and defines aX87_DOUBLE_ROUNDINGpreprocessor definition. No code currently uses this definition,but it’s available if anyone wishes to use it.(Added by Mark Dickinson;issue 2937.)

  configure also now sets aLDCXXSHARED Makefilevariable for supporting C++ linking. (Contributed by ArfreverFrehtes Taifersar Arahesis;issue 1222585.)

• The build process now creates the necessary files for pkg-configsupport. (Contributed by Clinton Roy;issue 3585.)

• The build process now supports Subversion 1.7. (Contributed byArfrever Frehtes Taifersar Arahesis;issue 6094.)

void *vtable;if (!PyCapsule_IsValid(capsule, "mymodule.CAPI") {        PyErr_SetString(PyExc_ValueError, "argument type invalid");        return NULL;}vtable = PyCapsule_GetPointer(capsule, "mymodule.CAPI");

Other Changes and Fixes¶

• Two benchmark scripts,iobench andccbench, wereadded to theTools directory.iobench measures thespeed of the built-in file I/O objects returned byopen()while performing various operations, andccbench is aconcurrency benchmark that tries to measure computing throughput,thread switching latency, and IO processing bandwidth whenperforming several tasks using a varying number of threads.
• TheTools/i18n/msgfmt.py script now understands pluralforms in.po files. (Fixed by Martin von Löwis;issue 5464.)
• When importing a module from a.pyc or.pyo filewith an existing.py counterpart, theco_filenameattributes of the resulting code objects are overwritten when theoriginal filename is obsolete. This can happen if the file has beenrenamed, moved, or is accessed through different paths. (Patch byŽiga Seilnacht and Jean-Paul Calderone;issue 1180193.)
• Theregrtest.py script now takes a--randseed=switch that takes an integer that will be used as the random seedfor the-r option that executes tests in random order.The-r option also reports the seed that was used(Added by Collin Winter.)
• Anotherregrtest.py switch is-j, whichtakes an integer specifying how many tests run in parallel. Thisallows reducing the total runtime on multi-core machines.This option is compatible with several other options, including the-R switch which is known to produce long runtimes.(Added by Antoine Pitrou,issue 6152.) This can also be usedwith a new-F switch that runs selected tests in a loopuntil they fail. (Added by Antoine Pitrou;issue 7312.)
• When executed as a script, thepy_compile.py module nowaccepts'-' as an argument, which will read standard input forthe list of filenames to be compiled. (Contributed by PiotrOżarowski;issue 8233.)

Porting to Python 2.7¶

This section lists previously described changes and other bugfixesthat may require changes to your code:

• Therange() function processes its arguments moreconsistently; it will now call__int__() on non-float,non-integer arguments that are supplied to it. (Fixed by AlexanderBelopolsky;issue 1533.)
• The stringformat() method changed the default precision usedfor floating-point and complex numbers from 6 decimalplaces to 12, which matches the precision used bystr().(Changed by Eric Smith;issue 5920.)
• Because of an optimization for thewith statement, the specialmethods__enter__() and__exit__() must belong to the object’stype, and cannot be directly attached to the object’s instance. Thisaffects new-style classes (derived fromobject) and C extensiontypes. (issue 6101.)
• Due to a bug in Python 2.6, theexc_value parameter to__exit__() methods was often the string representation of theexception, not an instance. This was fixed in 2.7, soexc_valuewill be an instance as expected. (Fixed by Florent Xicluna;issue 7853.)
• When a restricted set of attributes were set using__slots__,deleting an unset attribute would not raiseAttributeErroras you would expect. Fixed by Benjamin Peterson;issue 7604.)

In the standard library:

• Operations withdatetime instances that resulted in a yearfalling outside the supported range didn’t always raiseOverflowError. Such errors are now checked more carefullyand will now raise the exception. (Reported by Mark Leander, patchby Anand B. Pillai and Alexander Belopolsky;issue 7150.)

• When usingDecimal instances with a string’sformat() method, the default alignment was previouslyleft-alignment. This has been changed to right-alignment, which mightchange the output of your programs.(Changed by Mark Dickinson;issue 6857.)

  Comparisons involving a signaling NaN value (orsNAN) now signalInvalidOperation instead of silently returning a true orfalse value depending on the comparison operator. Quiet NaN values(orNaN) are now hashable. (Fixed by Mark Dickinson;issue 7279.)

• The ElementTree library,xml.etree, no longer escapesampersands and angle brackets when outputting an XML processinginstruction (which looks like<?xml-stylesheet href=”#style1”?>)or comment (which looks like<!– comment –>).(Patch by Neil Muller;issue 2746.)

• Thereadline() method ofStringIO objects now doesnothing when a negative length is requested, as other file-likeobjects do. (issue 7348).

• Thesyslog module will now use the value ofsys.argv[0] as theidentifier instead of the previous default value of'python'.(Changed by Sean Reifschneider;issue 8451.)

• Thetarfile module’s default error handling has changed, tono longer suppress fatal errors. The default error level was previously 0,which meant that errors would only result in a message being written to thedebug log, but because the debug log is not activated by default,these errors go unnoticed. The default error level is now 1,which raises an exception if there’s an error.(Changed by Lars Gustäbel;issue 7357.)

• Theurlparse module’surlsplit() now handlesunknown URL schemes in a fashion compliant withRFC 3986: if theURL is of the form"<something>://...", the text before the:// is treated as the scheme, even if it’s a made-up scheme thatthe module doesn’t know about. This change may break code thatworked around the old behaviour. For example, Python 2.6.4 or 2.5will return the following:

  >>>importurlparse>>>urlparse.urlsplit('invented://host/filename?query')('invented', '', '//host/filename?query', '', '')

  Python 2.7 (and Python 2.6.5) will return:

  >>>importurlparse>>>urlparse.urlsplit('invented://host/filename?query')('invented', 'host', '/filename?query', '', '')

  (Python 2.7 actually produces slightly different output, since itreturns a named tuple instead of a standard tuple.)

For C extensions:

• C extensions that use integer format codes with thePyArg_Parse*family of functions will now raise aTypeError exceptioninstead of triggering aDeprecationWarning (issue 5080).
• Use the newPyOS_string_to_double() function instead of the oldPyOS_ascii_strtod() andPyOS_ascii_atof() functions,which are now deprecated.

For applications that embed Python:

• ThePySys_SetArgvEx() function was added, lettingapplications close a security hole when the existingPySys_SetArgv() function was used. Check whether you’recallingPySys_SetArgv() and carefully consider whether theapplication should be usingPySys_SetArgvEx() withupdatepath set to false.

Acknowledgements¶

The author would like to thank the following people for offeringsuggestions, corrections and assistance with various drafts of thisarticle: Nick Coghlan, Philip Jenvey, Ryan Lovett, R. David Murray,Hugh Secker-Walker.