pyarrow.Codec#

classpyarrow.Codec(strcompression,compression_level=None)#

Bases:_Weakrefable

Compression codec.

Parameters:

compressionstr

Type of compression codec to initialize, valid values are: ‘gzip’,‘bz2’, ‘brotli’, ‘lz4’ (or ‘lz4_frame’), ‘lz4_raw’, ‘zstd’ and‘snappy’.

compression_levelint,None

Optional parameter specifying how aggressively to compress. Thepossible ranges and effect of this parameter depend on the specificcodec chosen. Higher values compress more but typically use moreresources (CPU/RAM). Some codecs support negative values.

gzip

The compression_level maps to the memlevel parameter ofdeflateInit2. Higher levels use more RAM but are fasterand should have higher compression ratios.

bz2

The compression level maps to the blockSize100k parameter ofthe BZ2_bzCompressInit function. Higher levels use more RAMbut are faster and should have higher compression ratios.

brotli

The compression level maps to the BROTLI_PARAM_QUALITYparameter. Higher values are slower and should have highercompression ratios.

lz4/lz4_frame/lz4_raw

The compression level parameter is not supported and mustbe None

zstd

The compression level maps to the compressionLevel parameterof ZSTD_initCStream. Negative values are supported. Highervalues are slower and should have higher compression ratios.

snappy

The compression level parameter is not supported and mustbe None

Raises:

ValueError

If invalid compression value is passed.

Examples

>>>importpyarrowaspa>>>pa.Codec.is_available('gzip')True>>>codec=pa.Codec('gzip')>>>codec.name'gzip'>>>codec.compression_level9

__init__(*args,**kwargs)#

Methods

__init__(*args, **kwargs)
compress(self, buf[, asbytes, memory_pool])	Compress data from buffer-like object.
decompress(self, buf[, decompressed_size, ...])	Decompress data from buffer-like object.
default_compression_level(str compression)	Returns the compression level that Arrow will use for the codec if None is specified.
detect(path)	Detect and instantiate compression codec based on file extension.
is_available(str compression)	Returns whether the compression support has been built and enabled.
maximum_compression_level(str compression)	Returns the largest valid value for the compression level
minimum_compression_level(str compression)	Returns the smallest valid value for the compression level
supports_compression_level(str compression)	Returns true if the compression level parameter is supported for the given codec.

Attributes

compression_level	Returns the compression level parameter of the codec
name	Returns the name of the codec

compress(self,buf,asbytes=False,memory_pool=None)#

Compress data from buffer-like object.

Parameters:

bufpyarrow.Buffer,bytes, or other object supporting buffer protocol

asbytesbool, defaultFalse

Return result as Python bytes object, otherwise Buffer

memory_poolMemoryPool, defaultNone

Memory pool to use for buffer allocations, if any

Returns:

compressedpyarrow.Buffer orbytes (if asbytes=True)

compression_level#

Returns the compression level parameter of the codec

decompress(self,buf,decompressed_size=None,asbytes=False,memory_pool=None)#

Decompress data from buffer-like object.

Parameters:

bufpyarrow.Buffer,bytes, or memoryview-compatible object

decompressed_sizeint, defaultNone

Size of the decompressed result

asbytesbool, defaultFalse

Return result as Python bytes object, otherwise Buffer

memory_poolMemoryPool, defaultNone

Memory pool to use for buffer allocations, if any.

Returns:

uncompressedpyarrow.Buffer orbytes (if asbytes=True)

staticdefault_compression_level(strcompression)#

Returns the compression level that Arrow will use for the codec ifNone is specified.

Parameters:

compressionstr

Type of compression codec,refer to Codec docstring for a list of supported ones.

staticdetect(path)#

Detect and instantiate compression codec based on file extension.

Parameters:

pathstr, path-like

File-path to detect compression from.

Returns:

Codec

Raises:

TypeError

If the passed value is not path-like.

ValueError

If the compression can’t be detected from the path.

staticis_available(strcompression)#

Returns whether the compression support has been built and enabled.

Parameters:

compressionstr

Type of compression codec,refer to Codec docstring for a list of supported ones.

Returns:

bool

staticmaximum_compression_level(strcompression)#

Returns the largest valid value for the compression level

Parameters:

compressionstr

Type of compression codec,refer to Codec docstring for a list of supported ones.

staticminimum_compression_level(strcompression)#

Returns the smallest valid value for the compression level

Parameters:

compressionstr

Type of compression codec,refer to Codec docstring for a list of supported ones.

name#

Returns the name of the codec

staticsupports_compression_level(strcompression)#

Returns true if the compression level parameter is supportedfor the given codec.

Parameters:

compressionstr

Type of compression codec,refer to Codec docstring for a list of supported ones.