13.6.1.TarFile Objects¶

TheTarFile object provides an interface to a tar archive. A tararchive is a sequence of blocks. An archive member (a stored file) is made up ofa header block followed by data blocks. It is possible to store a file in a tararchive several times. Each archive member is represented by aTarInfoobject, seeTarInfo Objects for details.

ATarFile object can be used as a context manager in awithstatement. It will automatically be closed when the block is completed. Pleasenote that in the event of an exception an archive opened for writing will notbe finalized; only the internally used file object will be closed. See theExamples section for a use case.

classtarfile.TarFile(name=None,mode='r',fileobj=None,format=DEFAULT_FORMAT,tarinfo=TarInfo,dereference=False,ignore_zeros=False,encoding=ENCODING,errors='surrogateescape',pax_headers=None,debug=0,errorlevel=0)

All following arguments are optional and can be accessed as instance attributesas well.

name is the pathname of the archive.name may be apath-like object.It can be omitted iffileobj is given.In this case, the file object’sname attribute is used if it exists.

mode is either'r' to read from an existing archive,'a' to appenddata to an existing file,'w' to create a new file overwriting an existingone, or'x' to create a new file only if it does not already exist.

Iffileobj is given, it is used for reading or writing data. If it can bedetermined,mode is overridden byfileobj’s mode.fileobj will be usedfrom position 0.

Note

fileobj is not closed, whenTarFile is closed.

format controls the archive format. It must be one of the constantsUSTAR_FORMAT,GNU_FORMAT orPAX_FORMAT that aredefined at module level.

Thetarinfo argument can be used to replace the defaultTarInfo classwith a different one.

Ifdereference isFalse, add symbolic and hard links to the archive. If itisTrue, add the content of the target files to the archive. This has noeffect on systems that do not support symbolic links.

Ifignore_zeros isFalse, treat an empty block as the end of the archive.If it isTrue, skip empty (and invalid) blocks and try to get as many membersas possible. This is only useful for reading concatenated or damaged archives.

debug can be set from0 (no debug messages) up to3 (all debugmessages). The messages are written tosys.stderr.

Iferrorlevel is0, all errors are ignored when usingTarFile.extract().Nevertheless, they appear as error messages in the debug output, when debuggingis enabled. If1, allfatal errors are raised asOSErrorexceptions. If2, allnon-fatal errors are raised asTarErrorexceptions as well.

Theencoding anderrors arguments define the character encoding to beused for reading or writing the archive and how conversion errors are goingto be handled. The default settings will work for most users.See sectionUnicode issues for in-depth information.

Thepax_headers argument is an optional dictionary of strings whichwill be added as a pax global header ifformat isPAX_FORMAT.

Changed in version 3.2:Use'surrogateescape' as the default for theerrors argument.

Changed in version 3.5:The'x' (exclusive creation) mode was added.

Changed in version 3.6:Thename parameter accepts apath-like object.

classmethodTarFile.open(...)¶

Alternative constructor. Thetarfile.open() function is actually ashortcut to this classmethod.

TarFile.getmember(name)¶

Return aTarInfo object for membername. Ifname can not be foundin the archive,KeyError is raised.

Note

If a member occurs more than once in the archive, its last occurrence is assumedto be the most up-to-date version.

TarFile.getmembers()¶

Return the members of the archive as a list ofTarInfo objects. Thelist has the same order as the members in the archive.

TarFile.getnames()¶

Return the members as a list of their names. It has the same order as the listreturned bygetmembers().

TarFile.list(verbose=True,*,members=None)¶

Print a table of contents tosys.stdout. Ifverbose isFalse,only the names of the members are printed. If it isTrue, outputsimilar to that ofls -l is produced. If optionalmembers isgiven, it must be a subset of the list returned bygetmembers().

Changed in version 3.5:Added themembers parameter.

TarFile.next()¶

Return the next member of the archive as aTarInfo object, whenTarFile is opened for reading. ReturnNone if there is no moreavailable.

TarFile.extractall(path=".",members=None,*,numeric_owner=False)¶

Extract all members from the archive to the current working directory ordirectorypath. If optionalmembers is given, it must be a subset of thelist returned bygetmembers(). Directory information like owner,modification time and permissions are set after all members have been extracted.This is done to work around two problems: A directory’s modification time isreset each time a file is created in it. And, if a directory’s permissions donot allow writing, extracting files to it will fail.

Ifnumeric_owner isTrue, the uid and gid numbers from the tarfileare used to set the owner/group for the extracted files. Otherwise, the namedvalues from the tarfile are used.

Warning

Never extract archives from untrusted sources without prior inspection.It is possible that files are created outside ofpath, e.g. membersthat have absolute filenames starting with"/" or filenames with twodots"..".

Changed in version 3.5:Added thenumeric_owner parameter.

Changed in version 3.6:Thepath parameter accepts apath-like object.

TarFile.extract(member,path="",set_attrs=True,*,numeric_owner=False)¶

Extract a member from the archive to the current working directory, using itsfull name. Its file information is extracted as accurately as possible.membermay be a filename or aTarInfo object. You can specify a differentdirectory usingpath.path may be apath-like object.File attributes (owner, mtime, mode) are set unlessset_attrs is false.

Ifnumeric_owner isTrue, the uid and gid numbers from the tarfileare used to set the owner/group for the extracted files. Otherwise, the namedvalues from the tarfile are used.

Note

Theextract() method does not take care of several extraction issues.In most cases you should consider using theextractall() method.

Warning

See the warning forextractall().

Changed in version 3.2:Added theset_attrs parameter.

Changed in version 3.5:Added thenumeric_owner parameter.

Changed in version 3.6:Thepath parameter accepts apath-like object.

TarFile.extractfile(member)¶

Extract a member from the archive as a file object.member may be a filenameor aTarInfo object. Ifmember is a regular file or a link, anio.BufferedReader object is returned. Otherwise,None isreturned.

Changed in version 3.3:Return anio.BufferedReader object.

TarFile.add(name,arcname=None,recursive=True,exclude=None,*,filter=None)¶

Add the filename to the archive.name may be any type of file(directory, fifo, symbolic link, etc.). If given,arcname specifies analternative name for the file in the archive. Directories are addedrecursively by default. This can be avoided by settingrecursive toFalse. Ifexclude is given, it must be a function that takes onefilename argument and returns a boolean value. Depending on this value therespective file is either excluded (True) or added(False). Iffilter is specified it must be a keyword argument. Itshould be a function that takes aTarInfo object argument andreturns the changedTarInfo object. If it instead returnsNone theTarInfo object will be excluded from thearchive. SeeExamples for an example.

Changed in version 3.2:Added thefilter parameter.

Deprecated since version 3.2:Theexclude parameter is deprecated, please use thefilter parameterinstead.

TarFile.addfile(tarinfo,fileobj=None)¶

Add theTarInfo objecttarinfo to the archive. Iffileobj is given,it should be abinary file, andtarinfo.size bytes are read from it and added to the archive. You cancreateTarInfo objects directly, or by usinggettarinfo().

TarFile.gettarinfo(name=None,arcname=None,fileobj=None)¶

Create aTarInfo object from the result ofos.stat() orequivalent on an existing file. The file is either named byname, orspecified as afile objectfileobj with a file descriptor.name may be apath-like object. Ifgiven,arcname specifies an alternative name for the file in thearchive, otherwise, the name is taken fromfileobj’sname attribute, or thename argument. The nameshould be a text string.

You can modifysome of theTarInfo’s attributes before you add it usingaddfile().If the file object is not an ordinary file object positioned at thebeginning of the file, attributes such assize may needmodifying. This is the case for objects such asGzipFile.Thename may also be modified, in which casearcnamecould be a dummy string.

Changed in version 3.6:Thename parameter accepts apath-like object.

TarFile.close()¶

Close theTarFile. In write mode, two finishing zero blocks areappended to the archive.

TarFile.pax_headers¶

A dictionary containing key-value pairs of pax global headers.

13.6.2.TarInfo Objects¶

ATarInfo object represents one member in aTarFile. Asidefrom storing all required attributes of a file (like file type, size, time,permissions, owner etc.), it provides some useful methods to determine its type.It doesnot contain the file’s data itself.

TarInfo objects are returned byTarFile’s methodsgetmember(),getmembers() andgettarinfo().

classtarfile.TarInfo(name="")¶

Create aTarInfo object.

classmethodTarInfo.frombuf(buf,encoding,errors)¶

Create and return aTarInfo object from string bufferbuf.

RaisesHeaderError if the buffer is invalid.

classmethodTarInfo.fromtarfile(tarfile)¶

Read the next member from theTarFile objecttarfile and return it asaTarInfo object.

TarInfo.tobuf(format=DEFAULT_FORMAT,encoding=ENCODING,errors='surrogateescape')¶

Create a string buffer from aTarInfo object. For information on thearguments see the constructor of theTarFile class.

Changed in version 3.2:Use'surrogateescape' as the default for theerrors argument.

ATarInfo object has the following public data attributes:

TarInfo.name¶

Name of the archive member.

TarInfo.size¶

Size in bytes.

TarInfo.mtime¶

Time of last modification.

TarInfo.mode¶

Permission bits.

TarInfo.type¶

File type.type is usually one of these constants:REGTYPE,AREGTYPE,LNKTYPE,SYMTYPE,DIRTYPE,FIFOTYPE,CONTTYPE,CHRTYPE,BLKTYPE,GNUTYPE_SPARSE. To determine the type of aTarInfo objectmore conveniently, use theis*() methods below.

TarInfo.linkname¶

Name of the target file name, which is only present inTarInfo objectsof typeLNKTYPE andSYMTYPE.

TarInfo.uid¶

User ID of the user who originally stored this member.

TarInfo.gid¶

Group ID of the user who originally stored this member.

TarInfo.uname¶

User name.

TarInfo.gname¶

Group name.

TarInfo.pax_headers¶

A dictionary containing key-value pairs of an associated pax extended header.

ATarInfo object also provides some convenient query methods:

TarInfo.isfile()¶

ReturnTrue if theTarinfo object is a regular file.

TarInfo.isreg()¶

Same asisfile().

TarInfo.isdir()¶

ReturnTrue if it is a directory.

TarInfo.issym()¶

ReturnTrue if it is a symbolic link.

TarInfo.islnk()¶

ReturnTrue if it is a hard link.

TarInfo.ischr()¶

ReturnTrue if it is a character device.

TarInfo.isblk()¶

ReturnTrue if it is a block device.

TarInfo.isfifo()¶

ReturnTrue if it is a FIFO.

TarInfo.isdev()¶

ReturnTrue if it is one of character device, block device or FIFO.

13.6.3.Command-Line Interface¶

Thetarfile module provides a simple command-line interface to interactwith tar archives.

If you want to create a new tar archive, specify its name after the-coption and then list the filename(s) that should be included:

$python-mtarfile-cmonty.tarspam.txteggs.txt

Passing a directory is also acceptable:

$python-mtarfile-cmonty.tarlife-of-brian_1979/

If you want to extract a tar archive into the current directory, usethe-e option:

$python-mtarfile-emonty.tar

You can also extract a tar archive into a different directory by passing thedirectory’s name:

$python-mtarfile-emonty.tarother-dir/

For a list of the files in a tar archive, use the-l option:

$python-mtarfile-lmonty.tar

13.6.4.Examples¶

How to extract an entire tar archive to the current working directory:

importtarfiletar=tarfile.open("sample.tar.gz")tar.extractall()tar.close()

How to extract a subset of a tar archive withTarFile.extractall() usinga generator function instead of a list:

importosimporttarfiledefpy_files(members):fortarinfoinmembers:ifos.path.splitext(tarinfo.name)[1]==".py":yieldtarinfotar=tarfile.open("sample.tar.gz")tar.extractall(members=py_files(tar))tar.close()

How to create an uncompressed tar archive from a list of filenames:

importtarfiletar=tarfile.open("sample.tar","w")fornamein["foo","bar","quux"]:tar.add(name)tar.close()

The same example using thewith statement:

importtarfilewithtarfile.open("sample.tar","w")astar:fornamein["foo","bar","quux"]:tar.add(name)

How to read a gzip compressed tar archive and display some member information:

importtarfiletar=tarfile.open("sample.tar.gz","r:gz")fortarinfointar:print(tarinfo.name,"is",tarinfo.size,"bytes in size and is",end="")iftarinfo.isreg():print("a regular file.")eliftarinfo.isdir():print("a directory.")else:print("something else.")tar.close()

How to create an archive and reset the user information using thefilterparameter inTarFile.add():

importtarfiledefreset(tarinfo):tarinfo.uid=tarinfo.gid=0tarinfo.uname=tarinfo.gname="root"returntarinfotar=tarfile.open("sample.tar.gz","w:gz")tar.add("foo",filter=reset)tar.close()

13.6.5.Supported tar formats¶

There are three tar formats that can be created with thetarfile module:

• The POSIX.1-1988 ustar format (USTAR_FORMAT). It supports filenamesup to a length of at best 256 characters and linknames up to 100 characters. Themaximum file size is 8 GiB. This is an old and limited but widelysupported format.

• The GNU tar format (GNU_FORMAT). It supports long filenames andlinknames, files bigger than 8 GiB and sparse files. It is the de factostandard on GNU/Linux systems.tarfile fully supports the GNU tarextensions for long names, sparse file support is read-only.

• The POSIX.1-2001 pax format (PAX_FORMAT). It is the most flexibleformat with virtually no limits. It supports long filenames and linknames, largefiles and stores pathnames in a portable way. However, not all tarimplementations today are able to handle pax archives properly.

  Thepax format is an extension to the existingustar format. It uses extraheaders for information that cannot be stored otherwise. There are two flavoursof pax headers: Extended headers only affect the subsequent file header, globalheaders are valid for the complete archive and affect all following files. Allthe data in a pax header is encoded inUTF-8 for portability reasons.

There are some more variants of the tar format which can be read, but notcreated:

• The ancient V7 format. This is the first tar format from Unix Seventh Edition,storing only regular files and directories. Names must not be longer than 100characters, there is no user/group name information. Some archives havemiscalculated header checksums in case of fields with non-ASCII characters.

• The SunOS tar extended format. This format is a variant of the POSIX.1-2001pax format, but is not compatible.

13.6.6.Unicode issues¶

The tar format was originally conceived to make backups on tape drives with themain focus on preserving file system information. Nowadays tar archives arecommonly used for file distribution and exchanging archives over networks. Oneproblem of the original format (which is the basis of all other formats) isthat there is no concept of supporting different character encodings. Forexample, an ordinary tar archive created on aUTF-8 system cannot be readcorrectly on aLatin-1 system if it contains non-ASCII characters. Textualmetadata (like filenames, linknames, user/group names) will appear damaged.Unfortunately, there is no way to autodetect the encoding of an archive. Thepax format was designed to solve this problem. It stores non-ASCII metadatausing the universal character encodingUTF-8.

The details of character conversion intarfile are controlled by theencoding anderrors keyword arguments of theTarFile class.

encoding defines the character encoding to use for the metadata in thearchive. The default value issys.getfilesystemencoding() or'ascii'as a fallback. Depending on whether the archive is read or written, themetadata must be either decoded or encoded. Ifencoding is not setappropriately, this conversion may fail.

Theerrors argument defines how characters are treated that cannot beconverted. Possible values are listed in sectionError Handlers.The default scheme is'surrogateescape' which Python also uses for itsfile system calls, seeFile Names, Command Line Arguments, and Environment Variables.

In case ofPAX_FORMAT archives,encoding is generally not neededbecause all the metadata is stored usingUTF-8.encoding is only used inthe rare cases when binary pax headers are decoded or when strings withsurrogate characters are stored.