10.1.Operating System Interface¶

Theos module provides dozens of functions for interacting with theoperating system:

>>>importos>>>os.getcwd()# Return the current working directory'C:\\Python313'>>>os.chdir('/server/accesslogs')# Change current working directory>>>os.system('mkdir today')# Run the command mkdir in the system shell0

Be sure to use theimportos style instead offromosimport*. Thiswill keepos.open() from shadowing the built-inopen() function whichoperates much differently.

The built-indir() andhelp() functions are useful as interactiveaids for working with large modules likeos:

>>>importos>>>dir(os)<returns a list of all module functions>>>>help(os)<returns an extensive manual page created from the module's docstrings>

For daily file and directory management tasks, theshutil module providesa higher level interface that is easier to use:

>>>importshutil>>>shutil.copyfile('data.db','archive.db')'archive.db'>>>shutil.move('/build/executables','installdir')'installdir'

10.2.File Wildcards¶

Theglob module provides a function for making file lists from directorywildcard searches:

>>>importglob>>>glob.glob('*.py')['primes.py', 'random.py', 'quote.py']

10.3.Command Line Arguments¶

Common utility scripts often need to process command line arguments. Thesearguments are stored in thesys module’sargv attribute as a list. Forinstance, let’s take the followingdemo.py file:

# File demo.pyimportsysprint(sys.argv)

Here is the output from runningpythondemo.pyonetwothree at the commandline:

['demo.py','one','two','three']

Theargparse module provides a more sophisticated mechanism to processcommand line arguments. The following script extracts one or more filenamesand an optional number of lines to be displayed:

importargparseparser=argparse.ArgumentParser(prog='top',description='Show top lines from each file')parser.add_argument('filenames',nargs='+')parser.add_argument('-l','--lines',type=int,default=10)args=parser.parse_args()print(args)

When run at the command line withpythontop.py--lines=5alpha.txtbeta.txt, the script setsargs.lines to5 andargs.filenamesto['alpha.txt','beta.txt'].

10.4.Error Output Redirection and Program Termination¶

Thesys module also has attributes forstdin,stdout, andstderr.The latter is useful for emitting warnings and error messages to make themvisible even whenstdout has been redirected:

>>>sys.stderr.write('Warning, log file not found starting a new one\n')Warning, log file not found starting a new one

The most direct way to terminate a script is to usesys.exit().

10.5.String Pattern Matching¶

There module provides regular expression tools for advanced stringprocessing. For complex matching and manipulation, regular expressions offersuccinct, optimized solutions:

>>>importre>>>re.findall(r'\bf[a-z]*','which foot or hand fell fastest')['foot', 'fell', 'fastest']>>>re.sub(r'(\b[a-z]+) \1',r'\1','cat in the the hat')'cat in the hat'

When only simple capabilities are needed, string methods are preferred becausethey are easier to read and debug:

>>>'tea for too'.replace('too','two')'tea for two'

10.6.Mathematics¶

Themath module gives access to the underlying C library functions forfloating-point math:

>>>importmath>>>math.cos(math.pi/4)0.70710678118654757>>>math.log(1024,2)10.0

Therandom module provides tools for making random selections:

>>>importrandom>>>random.choice(['apple','pear','banana'])'apple'>>>random.sample(range(100),10)# sampling without replacement[30, 83, 16, 4, 8, 81, 41, 50, 18, 33]>>>random.random()# random float from the interval [0.0, 1.0)0.17970987693706186>>>random.randrange(6)# random integer chosen from range(6)4

Thestatistics module calculates basic statistical properties(the mean, median, variance, etc.) of numeric data:

>>>importstatistics>>>data=[2.75,1.75,1.25,0.25,0.5,1.25,3.5]>>>statistics.mean(data)1.6071428571428572>>>statistics.median(data)1.25>>>statistics.variance(data)1.3720238095238095

The SciPy project <https://scipy.org> has many other modules for numericalcomputations.

10.7.Internet Access¶

There are a number of modules for accessing the internet and processing internetprotocols. Two of the simplest areurllib.request for retrieving datafrom URLs andsmtplib for sending mail:

>>>fromurllib.requestimporturlopen>>>withurlopen('http://worldtimeapi.org/api/timezone/etc/UTC.txt')asresponse:...forlineinresponse:...line=line.decode()# Convert bytes to a str...ifline.startswith('datetime'):...print(line.rstrip())# Remove trailing newline...datetime: 2022-01-01T01:36:47.689215+00:00>>>importsmtplib>>>server=smtplib.SMTP('localhost')>>>server.sendmail('soothsayer@example.org','jcaesar@example.org',..."""To: jcaesar@example.org...From: soothsayer@example.org......Beware the Ides of March....""")>>>server.quit()

(Note that the second example needs a mailserver running on localhost.)

10.8.Dates and Times¶

Thedatetime module supplies classes for manipulating dates and times inboth simple and complex ways. While date and time arithmetic is supported, thefocus of the implementation is on efficient member extraction for outputformatting and manipulation. The module also supports objects that are timezoneaware.

>>># dates are easily constructed and formatted>>>fromdatetimeimportdate>>>now=date.today()>>>nowdatetime.date(2003, 12, 2)>>>now.strftime("%m-%d-%y.%d %b %Y is a %A on the%d day of %B.")'12-02-03. 02 Dec 2003 is a Tuesday on the 02 day of December.'>>># dates support calendar arithmetic>>>birthday=date(1964,7,31)>>>age=now-birthday>>>age.days14368

10.9.Data Compression¶

Common data archiving and compression formats are directly supported by modulesincluding:zlib,gzip,bz2,lzma,zipfile andtarfile.

>>>importzlib>>>s=b'witch which has which witches wrist watch'>>>len(s)41>>>t=zlib.compress(s)>>>len(t)37>>>zlib.decompress(t)b'witch which has which witches wrist watch'>>>zlib.crc32(s)226805979

10.10.Performance Measurement¶

Some Python users develop a deep interest in knowing the relative performance ofdifferent approaches to the same problem. Python provides a measurement toolthat answers those questions immediately.

For example, it may be tempting to use the tuple packing and unpacking featureinstead of the traditional approach to swapping arguments. Thetimeitmodule quickly demonstrates a modest performance advantage:

>>>fromtimeitimportTimer>>>Timer('t=a; a=b; b=t','a=1; b=2').timeit()0.57535828626024577>>>Timer('a,b = b,a','a=1; b=2').timeit()0.54962537085770791

In contrast totimeit’s fine level of granularity, theprofile andpstats modules provide tools for identifying time critical sections inlarger blocks of code.

10.11.Quality Control¶

One approach for developing high quality software is to write tests for eachfunction as it is developed and to run those tests frequently during thedevelopment process.

Thedoctest module provides a tool for scanning a module and validatingtests embedded in a program’s docstrings. Test construction is as simple ascutting-and-pasting a typical call along with its results into the docstring.This improves the documentation by providing the user with an example and itallows the doctest module to make sure the code remains true to thedocumentation:

defaverage(values):"""Computes the arithmetic mean of a list of numbers.    >>> print(average([20, 30, 70]))    40.0    """returnsum(values)/len(values)importdoctestdoctest.testmod()# automatically validate the embedded tests

Theunittest module is not as effortless as thedoctest module,but it allows a more comprehensive set of tests to be maintained in a separatefile:

importunittestclassTestStatisticalFunctions(unittest.TestCase):deftest_average(self):self.assertEqual(average([20,30,70]),40.0)self.assertEqual(round(average([1,5,7]),1),4.3)withself.assertRaises(ZeroDivisionError):average([])withself.assertRaises(TypeError):average(20,30,70)unittest.main()# Calling from the command line invokes all tests

10.12.Batteries Included¶

Python has a “batteries included” philosophy. This is best seen through thesophisticated and robust capabilities of its larger packages. For example:

• Thexmlrpc.client andxmlrpc.server modules make implementingremote procedure calls into an almost trivial task. Despite the modules’names, no direct knowledge or handling of XML is needed.

• Theemail package is a library for managing email messages, includingMIME and otherRFC 2822-based message documents. Unlikesmtplib andpoplib which actually send and receive messages, the email package hasa complete toolset for building or decoding complex message structures(including attachments) and for implementing internet encoding and headerprotocols.

• Thejson package provides robust support for parsing thispopular data interchange format. Thecsv module supportsdirect reading and writing of files in Comma-Separated Value format,commonly supported by databases and spreadsheets. XML processing issupported by thexml.etree.ElementTree,xml.dom andxml.sax packages. Together, these modules and packagesgreatly simplify data interchange between Python applications andother tools.

• Thesqlite3 module is a wrapper for the SQLite databaselibrary, providing a persistent database that can be updated andaccessed using slightly nonstandard SQL syntax.

• Internationalization is supported by a number of modules includinggettext,locale, and thecodecs package.