Source code:Lib/tarfile.py
Thetarfile module makes it possible to read and write tararchives, including those using gzip, bz2 and lzma compression.Use thezipfile module to read or write.zip files, or thehigher-level functions inshutil.
Some facts and figures:
Changed in version 3.3:Added support forlzma compression.
Return aTarFile object for the pathnamename. For detailedinformation onTarFile objects and the keyword arguments that areallowed, seeTarFile Objects.
mode has to be a string of the form'filemode[:compression]', it defaultsto'r'. Here is a full list of mode combinations:
| mode | action |
|---|---|
| 'r'or'r:*' | Open for reading with transparentcompression (recommended). |
| 'r:' | Open for reading exclusively withoutcompression. |
| 'r:gz' | Open for reading with gzip compression. |
| 'r:bz2' | Open for reading with bzip2 compression. |
| 'r:xz' | Open for reading with lzma compression. |
| 'a'or'a:' | Open for appending with no compression. Thefile is created if it does not exist. |
| 'w'or'w:' | Open for uncompressed writing. |
| 'w:gz' | Open for gzip compressed writing. |
| 'w:bz2' | Open for bzip2 compressed writing. |
| 'w:xz' | Open for lzma compressed writing. |
Note that'a:gz','a:bz2' or'a:xz' is not possible. Ifmodeis not suitable to open a certain (compressed) file for reading,ReadError is raised. Usemode'r' to avoid this. If acompression method is not supported,CompressionError is raised.
Iffileobj is specified, it is used as an alternative to afile objectopened in binary mode forname. It is supposed to be at position 0.
For special purposes, there is a second format formode:'filemode|[compression]'.tarfile.open() will return aTarFileobject that processes its data as a stream of blocks. No random seeking willbe done on the file. If given,fileobj may be any object that has aread() orwrite() method (depending on themode).bufsizespecifies the blocksize and defaults to20*512 bytes. Use this variantin combination with e.g.sys.stdin, a socketfile object or a tapedevice. However, such aTarFile object is limited in that it doesnot allow to be accessed randomly, seeExamples. The currentlypossible modes:
| Mode | Action |
|---|---|
| 'r|*' | Open astream of tar blocks for readingwith transparent compression. |
| 'r|' | Open astream of uncompressed tar blocksfor reading. |
| 'r|gz' | Open a gzip compressedstream forreading. |
| 'r|bz2' | Open a bzip2 compressedstream forreading. |
| 'r|xz' | Open a lzma compressedstream forreading. |
| 'w|' | Open an uncompressedstream for writing. |
| 'w|gz' | Open a gzip compressedstream forwriting. |
| 'w|bz2' | Open a bzip2 compressedstream forwriting. |
| 'w|xz' | Open an lzma compressedstream forwriting. |
Class for reading and writing tar archives. Do not use this class directly,better usetarfile.open() instead. SeeTarFile Objects.
Thetarfile module defines the following exceptions:
Is raised when a tar archive is opened, that either cannot be handled by thetarfile module or is somehow invalid.
Is raised when a compression method is not supported or when the data cannot bedecoded properly.
Is raised for the limitations that are typical for stream-likeTarFileobjects.
Is raised fornon-fatal errors when usingTarFile.extract(), but only ifTarFile.errorlevel==2.
Is raised byTarInfo.frombuf() if the buffer it gets is invalid.
Each of the following constants defines a tar archive format that thetarfile module is able to create. See sectionSupported tar formats fordetails.
POSIX.1-1988 (ustar) format.
GNU tar format.
POSIX.1-2001 (pax) format.
The default format for creating archives. This is currentlyGNU_FORMAT.
The following variables are available on module level:
The default character encoding:'utf-8' on Windows,sys.getfilesystemencoding() otherwise.
See also
TheTarFile object provides an interface to a tar archive. A tararchive is a sequence of blocks. An archive member (a stored file) is made up ofa header block followed by data blocks. It is possible to store a file in a tararchive several times. Each archive member is represented by aTarInfoobject, seeTarInfo Objects for details.
ATarFile object can be used as a context manager in awithstatement. It will automatically be closed when the block is completed. Pleasenote that in the event of an exception an archive opened for writing will notbe finalized; only the internally used file object will be closed. See theExamples section for a use case.
New in version 3.2:Added support for the context manager protocol.
All following arguments are optional and can be accessed as instance attributesas well.
name is the pathname of the archive. It can be omitted iffileobj is given.In this case, the file object’sname attribute is used if it exists.
mode is either'r' to read from an existing archive,'a' to appenddata to an existing file or'w' to create a new file overwriting an existingone.
Iffileobj is given, it is used for reading or writing data. If it can bedetermined,mode is overridden byfileobj‘s mode.fileobj will be usedfrom position 0.
Note
fileobj is not closed, whenTarFile is closed.
format controls the archive format. It must be one of the constantsUSTAR_FORMAT,GNU_FORMAT orPAX_FORMAT that aredefined at module level.
Thetarinfo argument can be used to replace the defaultTarInfo classwith a different one.
Ifdereference isFalse, add symbolic and hard links to the archive. If itisTrue, add the content of the target files to the archive. This has noeffect on systems that do not support symbolic links.
Ifignore_zeros isFalse, treat an empty block as the end of the archive.If it isTrue, skip empty (and invalid) blocks and try to get as many membersas possible. This is only useful for reading concatenated or damaged archives.
debug can be set from0 (no debug messages) up to3 (all debugmessages). The messages are written tosys.stderr.
Iferrorlevel is0, all errors are ignored when usingTarFile.extract().Nevertheless, they appear as error messages in the debug output, when debuggingis enabled. If1, allfatal errors are raised asOSErrorexceptions. If2, allnon-fatal errors are raised asTarErrorexceptions as well.
Theencoding anderrors arguments define the character encoding to beused for reading or writing the archive and how conversion errors are goingto be handled. The default settings will work for most users.See sectionUnicode issues for in-depth information.
Changed in version 3.2:Use'surrogateescape' as the default for theerrors argument.
Thepax_headers argument is an optional dictionary of strings whichwill be added as a pax global header ifformat isPAX_FORMAT.
Alternative constructor. Thetarfile.open() function is actually ashortcut to this classmethod.
Return aTarInfo object for membername. Ifname can not be foundin the archive,KeyError is raised.
Note
If a member occurs more than once in the archive, its last occurrence is assumedto be the most up-to-date version.
Return the members of the archive as a list ofTarInfo objects. Thelist has the same order as the members in the archive.
Return the members as a list of their names. It has the same order as the listreturned bygetmembers().
Print a table of contents tosys.stdout. Ifverbose isFalse,only the names of the members are printed. If it isTrue, outputsimilar to that ofls -l is produced.
Return the next member of the archive as aTarInfo object, whenTarFile is opened for reading. ReturnNone if there is no moreavailable.
Extract all members from the archive to the current working directory ordirectorypath. If optionalmembers is given, it must be a subset of thelist returned bygetmembers(). Directory information like owner,modification time and permissions are set after all members have been extracted.This is done to work around two problems: A directory’s modification time isreset each time a file is created in it. And, if a directory’s permissions donot allow writing, extracting files to it will fail.
Warning
Never extract archives from untrusted sources without prior inspection.It is possible that files are created outside ofpath, e.g. membersthat have absolute filenames starting with"/" or filenames with twodots"..".
Extract a member from the archive to the current working directory, using itsfull name. Its file information is extracted as accurately as possible.membermay be a filename or aTarInfo object. You can specify a differentdirectory usingpath. File attributes (owner, mtime, mode) are set unlessset_attrs is false.
Note
Theextract() method does not take care of several extraction issues.In most cases you should consider using theextractall() method.
Warning
See the warning forextractall().
Changed in version 3.2:Added theset_attrs parameter.
Extract a member from the archive as a file object.member may be a filenameor aTarInfo object. Ifmember is a regular file or a link, anio.BufferedReader object is returned. Otherwise,None isreturned.
Changed in version 3.3:Return anio.BufferedReader object.
Add the filename to the archive.name may be any type of file(directory, fifo, symbolic link, etc.). If given,arcname specifies analternative name for the file in the archive. Directories are addedrecursively by default. This can be avoided by settingrecursive toFalse. Ifexclude is given, it must be a function that takes onefilename argument and returns a boolean value. Depending on this value therespective file is either excluded (True) or added(False). Iffilter is specified it must be a keyword argument. Itshould be a function that takes aTarInfo object argument andreturns the changedTarInfo object. If it instead returnsNone theTarInfo object will be excluded from thearchive. SeeExamples for an example.
Changed in version 3.2:Added thefilter parameter.
Deprecated since version 3.2:Theexclude parameter is deprecated, please use thefilter parameterinstead.
Add theTarInfo objecttarinfo to the archive. Iffileobj is given,tarinfo.size bytes are read from it and added to the archive. You cancreateTarInfo objects usinggettarinfo().
Note
On Windows platforms,fileobj should always be opened with mode'rb' toavoid irritation about the file size.
Create aTarInfo object for either the filename or thefileobjectfileobj (usingos.fstat() on its file descriptor). You can modifysome of theTarInfo‘s attributes before you add it usingaddfile().If given,arcname specifies an alternative name for the file in the archive.
Close theTarFile. In write mode, two finishing zero blocks areappended to the archive.
A dictionary containing key-value pairs of pax global headers.
ATarInfo object represents one member in aTarFile. Asidefrom storing all required attributes of a file (like file type, size, time,permissions, owner etc.), it provides some useful methods to determine its type.It doesnot contain the file’s data itself.
TarInfo objects are returned byTarFile‘s methodsgetmember(),getmembers() andgettarinfo().
Create and return aTarInfo object from string bufferbuf.
RaisesHeaderError if the buffer is invalid..
Read the next member from theTarFile objecttarfile and return it asaTarInfo object.
Create a string buffer from aTarInfo object. For information on thearguments see the constructor of theTarFile class.
Changed in version 3.2:Use'surrogateescape' as the default for theerrors argument.
ATarInfo object has the following public data attributes:
Name of the archive member.
Size in bytes.
Time of last modification.
Permission bits.
File type.type is usually one of these constants:REGTYPE,AREGTYPE,LNKTYPE,SYMTYPE,DIRTYPE,FIFOTYPE,CONTTYPE,CHRTYPE,BLKTYPE,GNUTYPE_SPARSE. To determine the type of aTarInfo objectmore conveniently, use theis_*() methods below.
Name of the target file name, which is only present inTarInfo objectsof typeLNKTYPE andSYMTYPE.
User ID of the user who originally stored this member.
Group ID of the user who originally stored this member.
User name.
Group name.
A dictionary containing key-value pairs of an associated pax extended header.
ATarInfo object also provides some convenient query methods:
How to extract an entire tar archive to the current working directory:
importtarfiletar=tarfile.open("sample.tar.gz")tar.extractall()tar.close()
How to extract a subset of a tar archive withTarFile.extractall() usinga generator function instead of a list:
importosimporttarfiledefpy_files(members):fortarinfoinmembers:ifos.path.splitext(tarinfo.name)[1]==".py":yieldtarinfotar=tarfile.open("sample.tar.gz")tar.extractall(members=py_files(tar))tar.close()
How to create an uncompressed tar archive from a list of filenames:
importtarfiletar=tarfile.open("sample.tar","w")fornamein["foo","bar","quux"]:tar.add(name)tar.close()
The same example using thewith statement:
importtarfilewithtarfile.open("sample.tar","w")astar:fornamein["foo","bar","quux"]:tar.add(name)
How to read a gzip compressed tar archive and display some member information:
importtarfiletar=tarfile.open("sample.tar.gz","r:gz")fortarinfointar:print(tarinfo.name,"is",tarinfo.size,"bytes in size and is",end="")iftarinfo.isreg():print("a regular file.")eliftarinfo.isdir():print("a directory.")else:print("something else.")tar.close()
How to create an archive and reset the user information using thefilterparameter inTarFile.add():
importtarfiledefreset(tarinfo):tarinfo.uid=tarinfo.gid=0tarinfo.uname=tarinfo.gname="root"returntarinfotar=tarfile.open("sample.tar.gz","w:gz")tar.add("foo",filter=reset)tar.close()
There are three tar formats that can be created with thetarfile module:
The POSIX.1-1988 ustar format (USTAR_FORMAT). It supports filenamesup to a length of at best 256 characters and linknames up to 100 characters. Themaximum file size is 8 GiB. This is an old and limited but widelysupported format.
The GNU tar format (GNU_FORMAT). It supports long filenames andlinknames, files bigger than 8 GiB and sparse files. It is the de factostandard on GNU/Linux systems.tarfile fully supports the GNU tarextensions for long names, sparse file support is read-only.
The POSIX.1-2001 pax format (PAX_FORMAT). It is the most flexibleformat with virtually no limits. It supports long filenames and linknames, largefiles and stores pathnames in a portable way. However, not all tarimplementations today are able to handle pax archives properly.
Thepax format is an extension to the existingustar format. It uses extraheaders for information that cannot be stored otherwise. There are two flavoursof pax headers: Extended headers only affect the subsequent file header, globalheaders are valid for the complete archive and affect all following files. Allthe data in a pax header is encoded inUTF-8 for portability reasons.
There are some more variants of the tar format which can be read, but notcreated:
The tar format was originally conceived to make backups on tape drives with themain focus on preserving file system information. Nowadays tar archives arecommonly used for file distribution and exchanging archives over networks. Oneproblem of the original format (which is the basis of all other formats) isthat there is no concept of supporting different character encodings. Forexample, an ordinary tar archive created on aUTF-8 system cannot be readcorrectly on aLatin-1 system if it contains non-ASCII characters. Textualmetadata (like filenames, linknames, user/group names) will appear damaged.Unfortunately, there is no way to autodetect the encoding of an archive. Thepax format was designed to solve this problem. It stores non-ASCII metadatausing the universal character encodingUTF-8.
The details of character conversion intarfile are controlled by theencoding anderrors keyword arguments of theTarFile class.
encoding defines the character encoding to use for the metadata in thearchive. The default value issys.getfilesystemencoding() or'ascii'as a fallback. Depending on whether the archive is read or written, themetadata must be either decoded or encoded. Ifencoding is not setappropriately, this conversion may fail.
Theerrors argument defines how characters are treated that cannot beconverted. Possible values are listed in sectionCodec Base Classes.The default scheme is'surrogateescape' which Python also uses for itsfile system calls, seeFile Names, Command Line Arguments, and Environment Variables.
In case ofPAX_FORMAT archives,encoding is generally not neededbecause all the metadata is stored usingUTF-8.encoding is only used inthe rare cases when binary pax headers are decoded or when strings withsurrogate characters are stored.
13.5.zipfile — Work with ZIP archives
Enter search terms or a module, class or function name.
