mmap — Memory-mapped file support¶

Memory-mapped file objects behave like bothbytearray and likefile objects. You can use mmap objects in most placeswherebytearray are expected; for example, you can use theremodule to search through a memory-mapped file. You can also change a singlebyte by doingobj[index]=97, or change a subsequence by assigning to aslice:obj[i1:i2]=b'...'. You can also read and write data starting atthe current file position, andseek() through the file to different positions.

A memory-mapped file is created by themmap constructor, which isdifferent on Unix and on Windows. In either case you must provide a filedescriptor for a file opened for update. If you wish to map an existing Pythonfile object, use itsfileno() method to obtain the correct value for thefileno parameter. Otherwise, you can open the file using theos.open() function, which returns a file descriptor directly (the filestill needs to be closed when done).

For both the Unix and Windows versions of the constructor,access may bespecified as an optional keyword parameter.access accepts one of fourvalues:ACCESS_READ,ACCESS_WRITE, orACCESS_COPY tospecify read-only, write-through or copy-on-write memory respectively, orACCESS_DEFAULT to defer toprot.access can be used on both Unixand Windows. Ifaccess is not specified, Windows mmap returns awrite-through mapping. The initial memory values for all three access typesare taken from the specified file. Assignment to anACCESS_READmemory map raises aTypeError exception. Assignment to anACCESS_WRITE memory map affects both memory and the underlying file.Assignment to anACCESS_COPY memory map affects memory but does notupdate the underlying file.

To map anonymous memory, -1 should be passed as the fileno along with the length.

classmmap.mmap(fileno,length,tagname=None,access=ACCESS_DEFAULT,offset=0,*,trackfd=True)¶

(Windows version) Mapslength bytes from the file specified by thefile descriptorfileno, and creates a mmap object. Iflength is largerthan the current size of the file, the file is extended to containlengthbytes. Iflength is0, the maximum length of the map is the currentsize of the file, except that if the file is empty Windows raises anexception (you cannot create an empty mapping on Windows).

tagname, if specified and notNone, is a string giving a tag name forthe mapping. Windows allows you to have many different mappings againstthe same file. If you specify the name of an existing tag, that tag isopened, otherwise a new tag of this name is created. If this parameter isomitted orNone, the mapping is created without a name. Avoiding theuse of thetagname parameter will assist in keeping your code portablebetween Unix and Windows.

offset may be specified as a non-negative integer offset. mmap referenceswill be relative to the offset from the beginning of the file.offsetdefaults to 0.offset must be a multiple of theALLOCATIONGRANULARITY.

Iftrackfd isFalse, the file handle corresponding tofileno willnot be duplicated, and the resultingmmap object will notbe associated with the map’s underlying file.This means that thesize() andresize()methods will fail.This mode is useful to limit the number of open file handles.The original file can be renamed (but not deleted) after closingfileno.

Changed in version 3.15:Thetrackfd parameter was added.

Raises anauditing eventmmap.__new__ with argumentsfileno,length,access,offset.

classmmap.mmap(fileno,length,flags=MAP_SHARED,prot=PROT_WRITE|PROT_READ,access=ACCESS_DEFAULT,offset=0,*,trackfd=True)

(Unix version) Mapslength bytes from the file specified by the filedescriptorfileno, and returns a mmap object. Iflength is0, themaximum length of the map will be the current size of the file whenmmap is called.

flags specifies the nature of the mapping.MAP_PRIVATE creates aprivate copy-on-write mapping, so changes to the contents of the mmapobject will be private to this process, andMAP_SHARED creates amapping that’s shared with all other processes mapping the same areas ofthe file. The default value isMAP_SHARED. Some systems haveadditional possible flags with the full list specified inMAP_* constants.

prot, if specified, gives the desired memory protection; the two mostuseful values arePROT_READ andPROT_WRITE, to specifythat the pages may be read or written.prot defaults toPROT_READ|PROT_WRITE.

access may be specified in lieu offlags andprot as an optionalkeyword parameter. It is an error to specify bothflags,prot andaccess. See the description ofaccess above for information on how touse this parameter.

offset may be specified as a non-negative integer offset. mmap referenceswill be relative to the offset from the beginning of the file.offsetdefaults to 0.offset must be a multiple ofALLOCATIONGRANULARITYwhich is equal toPAGESIZE on Unix systems.

Iftrackfd isFalse, the file descriptor specified byfileno willnot be duplicated, and the resultingmmap object will notbe associated with the map’s underlying file.This means that thesize() andresize()methods will fail.This mode is useful to limit the number of open file descriptors.

To ensure validity of the created memory mapping the file specifiedby the descriptorfileno is internally automatically synchronizedwith the physical backing store on macOS.

Changed in version 3.13:Thetrackfd parameter was added.

This example shows a simple way of usingmmap:

importmmap# write a simple example filewithopen("hello.txt","wb")asf:f.write(b"Hello Python!\n")withopen("hello.txt","r+b")asf:# memory-map the file, size 0 means whole filemm=mmap.mmap(f.fileno(),0)# read content via standard file methodsprint(mm.readline())# prints b"Hello Python!\n"# read content via slice notationprint(mm[:5])# prints b"Hello"# update content using slice notation;# note that new content must have same sizemm[6:]=b" world!\n"# ... and read again using standard file methodsmm.seek(0)print(mm.readline())# prints b"Hello  world!\n"# close the mapmm.close()

mmap can also be used as a context manager in awithstatement:

importmmapwithmmap.mmap(-1,13)asmm:mm.write(b"Hello world!")

Added in version 3.2:Context manager support.

The next example demonstrates how to create an anonymous map and exchangedata between the parent and child processes:

importmmapimportosmm=mmap.mmap(-1,13)mm.write(b"Hello world!")pid=os.fork()ifpid==0:# In a child processmm.seek(0)print(mm.readline())mm.close()

Raises anauditing eventmmap.__new__ with argumentsfileno,length,access,offset.

Memory-mapped file objects support the following methods:

close()¶

Closes the mmap. Subsequent calls to other methods of the object willresult in a ValueError exception being raised. This will not closethe open file.

closed¶

True if the file is closed.

Added in version 3.2.

find(sub[,start[,end]])¶

Returns the lowest index in the object where the subsequencesub isfound, such thatsub is contained in the range [start,end].Optional argumentsstart andend are interpreted as in slice notation.Returns-1 on failure.

Changed in version 3.5:Writablebytes-like object is now accepted.

flush([offset[,size]])¶

Flushes changes made to the in-memory copy of a file back to disk. Withoutuse of this call there is no guarantee that changes are written back beforethe object is destroyed. Ifoffset andsize are specified, onlychanges to the given range of bytes will be flushed to disk; otherwise, thewhole extent of the mapping is flushed.offset must be a multiple of thePAGESIZE orALLOCATIONGRANULARITY.

None is returned to indicate success. An exception is raised when thecall failed.

Changed in version 3.8:Previously, a nonzero value was returned on success; zero was returnedon error under Windows. A zero value was returned on success; anexception was raised on error under Unix.

Changed in version 3.15:Allow specifyingoffset withoutsize. Previously, bothoffsetandsize parameters were required together. Nowoffset can bespecified alone, and the flush operation will extend fromoffsetto the end of the mmap.

madvise(option[,start[,length]])¶

Send adviceoption to the kernel about the memory region beginning atstart and extendinglength bytes.option must be one of theMADV_* constants available on the system. Ifstart andlength are omitted, the entire mapping is spanned. Onsome systems (including Linux),start must be a multiple of thePAGESIZE.

Availability: Systems with themadvise() system call.

Added in version 3.8.

move(dest,src,count)¶

Copy thecount bytes starting at offsetsrc to the destination indexdest. If the mmap was created withACCESS_READ, then calls tomove will raise aTypeError exception.

read([n])¶

Return abytes containing up ton bytes starting from thecurrent file position. If the argument is omitted,None or negative,return all bytes from the current file position to the end of themapping. The file position is updated to point after the bytes that werereturned.

Changed in version 3.3:Argument can be omitted orNone.

read_byte()¶

Returns a byte at the current file position as an integer, and advancesthe file position by 1.

readline()¶

Returns a single line, starting at the current file position and up to thenext newline. The file position is updated to point after the bytes that werereturned.

resize(newsize)¶

Resizes the map and the underlying file, if any.

Resizing a map created withaccess ofACCESS_READ orACCESS_COPY, will raise aTypeError exception.Resizing a map created withtrackfd set toFalse,will raise aValueError exception.

On Windows: Resizing the map will raise anOSError if there are othermaps against the same named file. Resizing an anonymous map (ie against thepagefile) will silently create a new map with the original data copied overup to the length of the new size.

Availability: Windows and systems with themremap() system call.

Changed in version 3.11:Correctly fails if attempting to resize when another map is heldAllows resize against an anonymous map on Windows

rfind(sub[,start[,end]])¶

Returns the highest index in the object where the subsequencesub isfound, such thatsub is contained in the range [start,end].Optional argumentsstart andend are interpreted as in slice notation.Returns-1 on failure.

Changed in version 3.5:Writablebytes-like object is now accepted.

seek(pos[,whence])¶

Set the file’s current position.whence argument is optional anddefaults toos.SEEK_SET or0 (absolute file positioning); othervalues areos.SEEK_CUR or1 (seek relative to the currentposition) andos.SEEK_END or2 (seek relative to the file’s end).

Changed in version 3.13:Return the new absolute position instead ofNone.

seekable()¶

Return whether the file supports seeking, and the return value is alwaysTrue.

Added in version 3.13.

size()¶

Return the length of the file, which can be larger than the size of thememory-mapped area.For an anonymous mapping, return its size.

Changed in version 3.15:Anonymous mappings are now supported on Unix.

tell()¶

Returns the current position of the file pointer.

write(bytes)¶

Write the bytes inbytes into memory at the current position of thefile pointer and return the number of bytes written (never less thanlen(bytes), since if the write fails, aValueError will beraised). The file position is updated to point after the bytes thatwere written. If the mmap was created withACCESS_READ, thenwriting to it will raise aTypeError exception.

Changed in version 3.5:Writablebytes-like object is now accepted.

Changed in version 3.6:The number of bytes written is now returned.

write_byte(byte)¶

Write the integerbyte into memory at the currentposition of the file pointer; the file position is advanced by1. Ifthe mmap was created withACCESS_READ, then writing to it willraise aTypeError exception.