Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

bpo-26253: Add compressionlevel to tarfile stream#2962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
serhiy-storchaka merged 5 commits intopython:mainfromjarondl:tarfile-compress-26253
Jun 25, 2022
Merged
Show file tree
Hide file tree
Changes from1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
NextNext commit
bpo-26253: Add compressionlevel to tarfile stream
`tarfile` already accepts a compressionlevel argument for creatingfiles. This patch adds the same for stream-based tarfile usage.The default is 9, the value that was previously hard-coded.
  • Loading branch information
@jarondl
jarondl committedMay 8, 2022
commit89fa205d27819c4d4cd444abbfde76e0abe15cea
4 changes: 2 additions & 2 deletionsDoc/library/tarfile.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -98,8 +98,8 @@ Some facts and figures:
If *fileobj* is specified, it is used as an alternative to a :term:`file object`
opened in binary mode for *name*. It is supposed to be at position 0.

For modes ``'w:gz'``, ``'r:gz'``, ``'w:bz2'``, ``'r:bz2'``, ``'x:gz'``,
``'x:bz2'``, :func:`tarfile.open` accepts the keyword argument
For modes ``'w:gz'``, ``'x:gz'``, ``'w|gz'``, ``'w:bz2'``, ``'x:bz2'``,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Add theversionchanged directive.

Document the change in the What's New document.

Copy link
ContributorAuthor

@jarondljarondlAug 1, 2017
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

versionchanged - Done

``'w|bz2'``, :func:`tarfile.open` accepts the keyword argument
*compresslevel* (default ``9``) to specify the compression level of the file.

For modes ``'w:xz'`` and ``'x:xz'``, :func:`tarfile.open` accepts the
Expand Down
33 changes: 19 additions & 14 deletionsLib/tarfile.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -336,7 +336,8 @@ class _Stream:
_Stream is intended to be used only internally.
"""

def __init__(self, name, mode, comptype, fileobj, bufsize):
def __init__(self, name, mode, comptype, fileobj, bufsize,
compresslevel=9):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Since_Stream is intended to be used only internally, we are free to change the signature of its constructor. The default argument forcompresslevel is not needed. This just duplicates its value.

"""Construct a _Stream object.
"""
self._extfileobj = True
Expand All@@ -350,14 +351,15 @@ def __init__(self, name, mode, comptype, fileobj, bufsize):
fileobj = _StreamProxy(fileobj)
comptype = fileobj.getcomptype()

self.name= name or ""
self.mode= mode
self.name = name or ""
self.mode = mode
self.comptype = comptype
self.fileobj = fileobj
self.bufsize = bufsize
self.buf = b""
self.pos = 0
self.closed = False
self.fileobj = fileobj
self.bufsize = bufsize
self.compresslevel = compresslevel

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is it needed to savecompresslevel as an attribute? Wouldn't be enough to passcompresslevel as an argument to_init_write_gz()?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No it is not needed. My first approach was to pass it as an argument to_init_write_gz(), with a default value of 9 (not to change the signature). But then I got the sense that an attribute goes more naturally with the rest of the class. I can revert to the first approach if you prefer.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Other attributes are used in multiple_Stream methods or are needed for simulating a file object. Butcompresslevel is used only in the constructor (_init_write_gz() is a factored out part of the constructor). It is not needed after creating a_Stream object.

self.buf = b""
self.pos = 0
self.closed = False

try:
if comptype == "gz":
Expand All@@ -383,7 +385,7 @@ def __init__(self, name, mode, comptype, fileobj, bufsize):
self.cmp = bz2.BZ2Decompressor()
self.exception = OSError
else:
self.cmp = bz2.BZ2Compressor()
self.cmp = bz2.BZ2Compressor(self.compresslevel)

elif comptype == "xz":
try:
Expand DownExpand Up@@ -413,10 +415,11 @@ def __del__(self):
def _init_write_gz(self):
"""Initialize for writing with gzip compression.
"""
self.cmp = self.zlib.compressobj(9, self.zlib.DEFLATED,
-self.zlib.MAX_WBITS,
self.zlib.DEF_MEM_LEVEL,
0)
self.cmp = self.zlib.compressobj(self.compresslevel,
self.zlib.DEFLATED,
-self.zlib.MAX_WBITS,
self.zlib.DEF_MEM_LEVEL,
0)
timestamp = struct.pack("<L", int(time.time()))
self.__write(b"\037\213\010\010" + timestamp + b"\002\377")
if self.name.endswith(".gz"):
Expand DownExpand Up@@ -1649,7 +1652,9 @@ def not_compressed(comptype):
if filemode not in ("r", "w"):
raise ValueError("mode must be 'r' or 'w'")

stream = _Stream(name, filemode, comptype, fileobj, bufsize)
compresslevel = kwargs.pop("compresslevel", 9)
stream = _Stream(name, filemode, comptype, fileobj, bufsize,
compresslevel)
try:
t = cls(name, filemode, stream, **kwargs)
except:
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Adjustable compression level for tarfile streams.

[8]ページ先頭

©2009-2025 Movatter.jp