pyarrow.MemoryMappedFile#

classpyarrow.MemoryMappedFile#

Bases:NativeFile

A stream that represents a memory-mapped file.

Supports ‘r’, ‘r+’, ‘w’ modes.

Examples

Create a new file with memory map:

>>>importpyarrowaspa>>>mmap=pa.create_memory_map('example_mmap.dat',10)>>>mmap<pyarrow.MemoryMappedFile closed=False own_file=False is_seekable=True is_writable=True is_readable=True>>>>mmap.close()

Open an existing file with memory map:

>>>withpa.memory_map('example_mmap.dat')asmmap:...mmap...<pyarrow.MemoryMappedFile closed=False own_file=False is_seekable=True is_writable=False is_readable=True>

__init__(*args,**kwargs)#

Methods

__init__(*args, **kwargs)
close(self)
create(path, size)	Create a MemoryMappedFile
download(self, stream_or_path[, buffer_size])	Read this file completely to a local path or destination stream.
fileno(self)
flush(self)	Flush the stream, if applicable.
get_stream(self, file_offset, nbytes)	Return an input stream that reads a file segment independent of the state of the file.
isatty(self)
metadata(self)	Return file metadata
read(self[, nbytes])	Read and return up to n bytes.
read1(self[, nbytes])	Read and return up to n bytes.
read_at(self, nbytes, offset)	Read indicated number of bytes at offset from the file
read_buffer(self[, nbytes])	Read from buffer.
readable(self)
readall(self)
readinto(self, b)	Read into the supplied buffer
readline(self[, size])	NOT IMPLEMENTED.
readlines(self[, hint])	NOT IMPLEMENTED.
resize(self, new_size)	Resize the map and underlying file.
seek(self, int64_t position, int whence=0)	Change current file stream position
seekable(self)
size(self)	Return file size
tell(self)	Return current stream position
truncate(self)	NOT IMPLEMENTED
upload(self, stream[, buffer_size])	Write from a source stream to this file.
writable(self)
write(self, data)	Write data to the file.
writelines(self, lines)	Write lines to the file.

Attributes

closed
mode	The file mode.

close(self)#

closed#

staticcreate(path,size)#

Create a MemoryMappedFile

Parameters:

pathstr

Where to create the file.

sizeint

Size of the memory mapped file.

download(self,stream_or_path,buffer_size=None)#

Read this file completely to a local path or destination stream.

This method first seeks to the beginning of the file.

Parameters:

stream_or_pathstr or file-like object

If a string, a local file path to write to; otherwise,should be a writable stream.

buffer_sizeint, optional

The buffer size to use for data transfers.

fileno(self)#

flush(self)#

Flush the stream, if applicable.

An error is raised if stream is not writable.

get_stream(self,file_offset,nbytes)#

Return an input stream that reads a file segment independent of thestate of the file.

Allows reading portions of a random access file as an input streamwithout interfering with each other.

Parameters:

file_offsetint

nbytesint

Returns:

streamNativeFile

isatty(self)#

metadata(self)#

Return file metadata

mode#

The file mode. Currently instances of NativeFile may support:

• rb: binary read

• wb: binary write

• rb+: binary read and write

• ab: binary append

read(self,nbytes=None)#

Read and return up to n bytes.

Ifnbytes is None, then the entire remaining file contents are read.

Parameters:

nbytesint, defaultNone

Returns:

databytes

read1(self,nbytes=None)#

Read and return up to n bytes.

Unlike read(), ifnbytes is None then a chunk is read, not theentire file.

Parameters:

nbytesint, defaultNone

The maximum number of bytes to read.

Returns:

databytes

read_at(self,nbytes,offset)#

Read indicated number of bytes at offset from the file

Parameters:

nbytesint

offsetint

Returns:

databytes

read_buffer(self,nbytes=None)#

Read from buffer.

Parameters:

nbytesint, optional

maximum number of bytes read

readable(self)#

readall(self)#

readinto(self,b)#

Read into the supplied buffer

Parameters:

bbuffer-like object

A writable buffer object (such as a bytearray).

Returns:

writtenint

number of bytes written

readline(self,size=None)#

NOT IMPLEMENTED. Read and return a line of bytes from the file.

If size is specified, read at most size bytes.

Line terminator is always b”n”.

Parameters:

sizeint

maximum number of bytes read

readlines(self,hint=None)#

NOT IMPLEMENTED. Read lines of the file

Parameters:

hintint

maximum number of bytes read until we stop

resize(self,new_size)#

Resize the map and underlying file.

Parameters:

new_sizenewsizeinbytes

seek(self,int64_tposition,intwhence=0)#

Change current file stream position

Parameters:

positionint

Byte offset, interpreted relative to value of whence argument

whenceint, default 0

Point of reference for seek offset

Returns:

int

The new absolute stream position.

Notes

Values of whence:* 0 – start of stream (the default); offset should be zero or positive* 1 – current stream position; offset may be negative* 2 – end of stream; offset is usually negative

seekable(self)#

size(self)#

Return file size

tell(self)#

Return current stream position

truncate(self)#

NOT IMPLEMENTED

upload(self,stream,buffer_size=None)#

Write from a source stream to this file.

Parameters:

streamfile-like object

Source stream to pipe to this file.

buffer_sizeint, optional

The buffer size to use for data transfers.

writable(self)#

write(self,data)#

Write data to the file.

Parameters:

databytes-like object orexporter of buffer protocol

Returns:

int

nbytes: number of bytes written

writelines(self,lines)#

Write lines to the file.

Parameters:

linesiterable

Iterable of bytes-like objects or exporters of buffer protocol