Setting a precedent

If and when other libraries for archive extraction, such aszipfile,gain similar functionality, they should mimic this API as much as it’sreasonable.

To enable this for simple cases, the built-in filters will have string names;e.g. users can passfilter='data' instead of a specific function that dealswithTarInfo objects.

Theshutil.unpack_archive() function will get afilter argument, which it will pass toextractall.

Adding function-based API that would work across archive formats isout of scope of this PEP.

Full disclosure & redistributor info

The PEP author works for Red Hat, a redistributor of Python with differentsecurity needs and support periods than CPython in general.Such redistributors may want to carry vendor patches to:

• Allow configuring the defaults system-wide, and
• Change the default as soon as possible, even in older Python versions.

The proposal makes this easy to do, and it allows users to querythe settings.

Modifying and forgetting member metadata

TheTarInfo class will gain a new method,replace(), which will work similarly todataclasses.replace.It will return a copy of theTarInfo object with attributesreplaced as specified by keyword-only arguments:

• name
• mtime
• mode
• linkname
• uid
• gid
• uname
• gname

Any of these, exceptname andlinkname, will be allowed to be settoNone.Whenextract orextractall encounters such aNone, it will notset that piece of metadata.(Ifuname orgname isNone, it will fall back touid orgidas if the name wasn’t found.)Whenaddfile ortobuf encounters such aNone, it will raise aValueError.Whenlist encounters such aNone, it will print a placeholder string.

The documentation will mention why the method is there:TarInfo objects retrieved fromTarFile.getmembersare “live”; modifying them directly will affect subsequent unrelatedoperations.

Filters

TarFile.extract andTarFile.extractall methodswill grow afilter keyword-only parameter,which takes a callable that can be called as:

filter(/,member:TarInfo,path:str)->TarInfo|None

wheremember is the member to be extracted, andpath is the path towhere the archive is extracted (i.e., it’ll be the same for every member).

When used it will be called on each member as it is extracted,and extraction will work with the result.If it returnsNone, the member will be skipped.

The function can also raise an exception.This can, depending onTarFile.errorlevel,abort the extraction or cause the member to be skipped.

We will also provide a set of defaults for common use cases.In addition to a function, thefilter argument can be oneof the following strings:

• 'fully_trusted': Current behavior: honor the metadata as is.Should be used if the user trusts the archive completely, or implements theirown complex verification.
• 'tar': Roughly follow defaults of the GNUtar command(when run as a normal user):
  • Strip leading'/' andos.sep from filenames
  • Refuse to extract files with absolute paths (after the/ strippingabove, e.g.C:/foo on Windows).
  • Refuse to extract files whose absolute path (after following symlinks)would end up outside the destination.(Note that GNUtar instead delays creating some links.)
  • Clear high mode bits (setuid, setgid, sticky) and group/other write bits(S_IWGRP|S_IWOTH).(This is an approximation of GNUtar’s default, which limits the modeby the currentumask setting.)
• 'data': Extract a “data” archive, disallowing common attack vectorsbut limiting functionality.In particular, many features specific to UNIX-style filesystems (orequivalently, to thetar archive format) are ignored, making this a goodfilter for cross-platform archives.In addition totar:
  • Refuse to extract links (hard or soft) that link to absolute paths.
  • Refuse to extract links (hard or soft) which end up linking to a pathoutside of the destination.(On systems that don’t support links,tarfile will, in most cases,fall back to creating regular files.This proposal doesn’t change that behaviour.)
  • Refuse to extract device files (including pipes).
  • For regular files and hard links:
    • Set the owner read and write permissions (S_IRUSR|S_IWUSR).
    • Remove the group & otherexecutable permission (S_IXGRP|S_IXOTH)if the owner doesn’t have it (S_IXUSR).
  • For other files (directories), ignore mode entirely (set it toNone).
  • Ignore user and group info (setuid,gid,uname,gnametoNone).

Any other string will cause aValueError.

The corresponding filter functions will be available astarfile.fully_trusted_filter(),tarfile.tar_filter(), etc., sothey can be easily used in custom policies.

Note that these filters never returnNone.Skipping members this way is a feature for user-defined filters.

Defaults and their configuration

TarFile will gain a new attribute,extraction_filter, to allow configuring the default filter.By default it will beNone, but users can set it to a callablethat will be used if thefilter argument is missing orNone.

If both the argument and attribute areNone:

• In Python 3.12-3.13, aDeprecationWarning will be emitted andextraction will use the'fully_trusted' filter.
• In Python 3.14+, it will use the'data' filter.

Applications and system integrators may wish to changeextraction_filterof theTarFile class itself to set a global default.When using a function, they will generally want to wrap it instaticmethod()to prevent injection of aself argument.

Subclasses ofTarFile can also overrideextraction_filter.

FilterError

A new exception,FilterError, will be added to thetarfilemodule.It’ll have several new subclasses, one for each of the refusal reasons above.FilterError’smember attribute will contain the relevantTarInfo.

In the lists above, “refusing” to extract a file means that aFilterErrorwill be raised.As with other extraction errors, if theTarFile.errorlevelis 1 or more, this will abort the extraction; witherrorlevel=0 the errorwill be logged and the member will be ignored, but extraction will continue.Note thatextractall() may leave the archive partially extracted;it is the user’s responsibility to clean up.

Errorlevel, and fatal/non-fatal errors

Currently,TarFile has anerrorlevelargument/attribute, which specifies how errors are handled:

• Witherrorlevel=0, documentation says that “all errors are ignoredwhen usingextract() andextractall()”.The code only ignoresnon-fatal andfatal errors (see below),so, for example, you still getTypeError if you passNone as thedestination path.
• Witherrorlevel=1 (the default), allnon-fatal errors are ignored.(They may be logged tosys.stderr by setting thedebugargument/attribute.)Which errors arenon-fatal is not defined in documentation, but code treatsExtractionError as such. Specifically, it’s these issues:
  • “unable to resolve link inside archive” (raised on systems that do notsupport symlinks)
  • “fifo/special devices not supported by system” (not used for failures ifthe system supports these, e.g. for aPermissionError)
  • “could not change owner/mode/modification time”

  Note that, for example,file name too long orout of disk space don’tqualify.Thenon-fatal errors are not very likely to appear on a Unix-like system.

• Witherrorlevel=2, all errors are raised, includingfatal ones.Which errors arefatal is, again, not defined; in practice it’sOSError.

A filter refusing to extract a member does not fit neatly into thefatal/non-fatal categories.

• This PEP does not change existing behavior. (Ideas for improvements arewelcome inDiscourse topic 25970.)
• When a filter refuses to extract a member, the error should not passsilently by default.

To satisfy this,FilterError will be considered afatal error, that is,it’ll be ignored only witherrorlevel=0.

Users that want to ignoreFilterError but not otherfatal errors shouldcreate a custom filter function, and call another filter in atry block.

Hints for further verification

Even with the proposed changes,tarfile will not besuited for extracting untrusted files without prior inspection.Among other issues, the proposed policies don’t prevent denial-of-serviceattacks.Users should do additional checks.

New docs will tell users to consider:

• extracting to a new empty directory,
• using external (e.g. OS-level) limits on disk, memory and CPU usage,
• checking filenames against an allow-list of characters (to filter out controlcharacters, confusables, etc.),
• checking that filenames have expected extensions (discouraging files thatexecute when you “click on them”, or extension-less files like Windowsspecial device names),
• limiting the number of extracted files, total size of extracted data,and size of individual files,
• checking for files that would be shadowed on case-insensitive filesystems.

Also, the docs will note that:

• tar files commonly contain multiple versions of the same file: later ones areexpected to overwrite earlier ones on extraction,
• tarfile does not protect against issues with “live” data, e.g. an attackertinkering with the destination directory while extracting (or adding) isgoing on (see theGNU tar manualfor more info).

This list is not comprehensive, but the documentation is a good place tocollect such general tips.It can be moved into a separate document if grows too long or if it needs tobe consolidated withzipfile orshutil(which is out of scope for this proposal).

TarInfo identity, andoffset

With filters that usereplace(), theTarInfo objects handledby the extraction machinery will not necessarily be the same objectsas those present inmembers.This may affectTarInfo subclasses that override methods likemakelink and rely on object identity.

Such code can switch to comparingoffset, the position of the memberheader inside the file.

Note that both the overridable methods andoffset are onlydocumented in source comments.

tarfile CLI

The CLI (python-mtarfile) will gain a--filter optionthat will take the name of one of the provided default filters.It won’t be possible to specify a custom filter function.

If--filter is not given, the CLI will use the default filter('fully_trusted' with a deprecation warning now, and'data' fromPython 3.14 on).

There will be no short option. (-f would be confusingly similar tothe filename option of GNUtar.)

Other archive libraries

If and when other archive libraries, such aszipfile,grow similar functionality, their extraction functions should use afilterargument that takes, at least, the strings'fully_trusted' (which shoulddisable any security precautions) and'data' (which should avoid featuresthat might surprise users).

Standardizing a function-based filter API is out of scope of this PEP.

Shutil

shutil.unpack_archive() will gain afilter argument.If it’s given, it will be passed to the underlying extraction function.Passing it for azip archive will fail for now (untilzipfilegains afilter argument, if it ever does).

Iffilter is not specified (or left asNone), it won’t be passedon, so extracting a tarball will use the default filter('fully_trusted' with a deprecation warning now, and'data' fromPython 3.14 on).

Complex filters

Note that some user-defined filters need, for example,to count extracted members of do post-processing.This requires a more complex API than afilter callable.However, that complex API need not be exposed totarfile.For example, with a hypotheticalStatefulFilter users would write:

withStatefulFilter()asfilter_func:my_tar.extract(path,filter=filter_func)

A simpleStatefulFilter example will be added to the docs.

SafeTarFile

An initial idea from Lars Gustäbel was to provide a separate class thatimplements security checks (seegh-65308).There are two major issues with this approach:

• The name is misleading. General archive operations can never be made “safe”from all kinds of unwanted behavior, without impacting legitimate use cases.
• It does not solve the problem of unsafe defaults.

However, many of the ideas behind SafeTarFile were reused in this PEP.

Add absolute_path option to tarfile

Issuegh-73974 asks for adding anabsolute_path option to extractionmethods. This would be a minimal change to formally resolveCVE-2007-4559.It doesn’t go far enough to protect the unaware, nor to empower the diligentand curious.

Other names for the'tar' filter

The'tar' filter exposes features specific to UNIX-like filesystems,so it could be named'unix'.Or'unix-like','nix','*nix','posix'?

Feature-wise,tar format andUNIX-like filesystem are essentiallyequivalent, sotar is a good name.

Adding filters to zipfile and shutil.unpack_archive

For consistency,zipfile andshutil.unpack_archive() could gain supportfor afilter argument.However, this would require research that this PEP’s author can’t promisefor Python 3.12.

Filters forzipfile would probably not help security.Zip is used primarily for cross-platform data bundles, and correspondingly,ZipFile.extract’s defaultsare already similar to what a'data' filter would do.A'fully_trusted' filter, which wouldnewly allow absolute paths and.. path components, might not be useful for much excepta unifiedunpack_archive API.

Filters should be useful for use cases other than security, but thosewould usually need custom filter functions, and those would need API that workswith bothTarInfo andZipInfo.That isdefinitely out of scope of this PEP.

If only this PEP is implemented and nothing changes forzipfile,the effect for callers ofunpack_archive is that the defaultfortar files is changing from'fully_trusted' tothe more appropriate'data'.In the interim period, Python 3.12-3.13 will emitDeprecationWarning.That’s annoying, but there are several ways to handle it: e.g. add afilter argument conditionally, setTarFile.extraction_filterglobally, or ignore/suppress the warning until Python 3.14.

Also, since many calls tounpack_archive are likely to be unsafe,there’s hope that theDeprecationWarning will often turn out to bea helpful hint to review affected code.