pyarrow.output_stream#

pyarrow.output_stream(source,compression='detect',buffer_size=None)#

Create an Arrow output stream.

Parameters:

sourcestr,Path, buffer, file-like object

The source to open for writing.

compressionstr optional, default ‘detect’

The compression algorithm to use for on-the-fly compression.If “detect” and source is a file path, then compression will bechosen based on the file extension.If None, no compression will be applied.Otherwise, a well-known algorithm name must be supplied (e.g. “gzip”).

buffer_sizeint, defaultNone

If None or 0, no buffering will happen. Otherwise the size of thetemporary write buffer.

Examples

Create a writable NativeFile from a pyarrow Buffer:

>>>importpyarrowaspa>>>data=b"buffer data">>>empty_obj=bytearray(11)>>>buf=pa.py_buffer(empty_obj)>>>withpa.output_stream(buf)asstream:...stream.write(data)...11>>>withpa.input_stream(buf)asstream:...stream.read(6)...b'buffer'

or from a memoryview object:

>>>buf=memoryview(empty_obj)>>>withpa.output_stream(buf)asstream:...stream.write(data)...11>>>withpa.input_stream(buf)asstream:...stream.read()...b'buffer data'

Create a writable NativeFile from a string or file path:

>>>withpa.output_stream('example_second.txt')asstream:...stream.write(b'Write some data')...15>>>withpa.input_stream('example_second.txt')asstream:...stream.read()...b'Write some data'