Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.4k
gh-78502: Add a trackfd parameter to mmap.mmap()#25425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Changes fromall commits
d42762c
8d141ad
12705a6
42302bc
9443f1f
eb8d798
8f05c7b
b207558
7607593
2d90229
0a717f8
7364a27
eca802c
51bb4e5
d274952
e33a7fd
File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -48,7 +48,7 @@ update the underlying file. | ||
To map anonymous memory, -1 should be passed as the fileno along with the length. | ||
.. class:: mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT, offset=0) | ||
**(Windows version)** Maps *length* bytes from the file specified by the | ||
file handle *fileno*, and creates a mmap object. If *length* is larger | ||
@@ -71,7 +71,8 @@ To map anonymous memory, -1 should be passed as the fileno along with the length | ||
.. audit-event:: mmap.__new__ fileno,length,access,offset mmap.mmap | ||
.. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, \ | ||
access=ACCESS_DEFAULT, offset=0, *, trackfd=True) | ||
:noindex: | ||
**(Unix version)** Maps *length* bytes from the file specified by the file | ||
@@ -102,10 +103,20 @@ To map anonymous memory, -1 should be passed as the fileno along with the length | ||
defaults to 0. *offset* must be a multiple of :const:`ALLOCATIONGRANULARITY` | ||
which is equal to :const:`PAGESIZE` on Unix systems. | ||
If *trackfd* is ``False``, the file descriptor specified by *fileno* will | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. I'd love to have some idea of why I might want to use this parameter. Right now it only describes the downsides. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. On Windows, the internally duplicated handle probably references an open that lacks delete access. It thus prevents deleting the file, even if the mapped section otherwise allows it (e.g. the section is mapped readonly). For example: >>> f=open('spam.txt')>>> m= mmap.mmap(f.fileno(),0,access=mmap.ACCESS_READ)>>> f.close()>>> os.remove('spam.txt')Traceback (most recent call last): File "<stdin>", line 1, in <module>PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'spam.txt'>>># I manually closed the internal handle via Process Explorer.>>> os.remove('spam.txt')>>> m[:]b'spam' There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. That sounds like a reason to at least add the argument for all platforms, which I'm generally in favour of anyway. It can have more appropriate semantics on Windows if needed (i.e. "doesn't hold an extra HANDLE" rather than "FD"). It's probably actually pretty useful to be able to immediately delete the file but keep the mapping open (which will keep the file on disk on Windows at least, so you can't reuse the name while it's in use). And it looks like the mapping doesn't lock out deletes, so I guess it'll work as intended. I'm not going to hold up this PR for it though. All I'll say is that if we ever do add that option, it should be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more.
I think
NTFS supports POSIX delete, in which a deleted file gets renamed to a reserved system directory until all references to the file object have been closed. That includes the internal pointer reference to a file object that's held by the memory manager for the mapped section. The internal file reference doesn't count toward the file's share mode, i.e. a memory-mapped file can be deleted even if the source open didn't share delete access. Actually, I just checked that the delete is allowed nowadays even if the mapped section has write access to the file, so my assumption was wrong that it would only work for a readonly mapping. You can observe this in Process Explorer. Switch the lower-pane view to DLLs (file- and pagefile-backed memory mappings), and add the name and path columns to the view. You'll see that the backing file gets moved to the "\$Extend\$Deleted" system directory on the volume after the file is 'deleted'. | ||
not be duplicated, and the resulting :class:`!mmap` object will not | ||
be associated with the map's underlying file. | ||
This means that the :meth:`~mmap.mmap.size` and :meth:`~mmap.mmap.resize` | ||
methods will fail. | ||
This mode is useful to limit the number of open file descriptors. | ||
To ensure validity of the created memory mapping the file specified | ||
by the descriptor *fileno* is internally automatically synchronized | ||
with the physical backing store on macOS. | ||
.. versionchanged:: 3.13 | ||
The *trackfd* parameter was added. | ||
This example shows a simple way of using :class:`~mmap.mmap`:: | ||
import mmap | ||
@@ -254,9 +265,12 @@ To map anonymous memory, -1 should be passed as the fileno along with the length | ||
.. method:: resize(newsize) | ||
Resizes the map and the underlying file, if any. | ||
Resizing a map created with *access* of :const:`ACCESS_READ` or | ||
:const:`ACCESS_COPY`, will raise a :exc:`TypeError` exception. | ||
Resizing a map created with with *trackfd* set to ``False``, | ||
will raise a :exc:`ValueError` exception. | ||
**On Windows**: Resizing the map will raise an :exc:`OSError` if there are other | ||
maps against the same named file. Resizing an anonymous map (ie against the | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
:class:`mmap.mmap` now has a *trackfd* parameter on Unix; if it is | ||
``False``, the file descriptor specified by *fileno* will not be duplicated. |