
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2018-06-15 18:14 byserhiy.storchaka, last changed2022-04-11 14:59 byadmin. This issue is nowclosed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 7931 | merged | serhiy.storchaka,2018-06-26 16:46 | |
| PR 8583 | merged | miss-islington,2018-07-31 07:25 | |
| PR 8584 | merged | serhiy.storchaka,2018-07-31 07:38 | |
| Messages (8) | |||
|---|---|---|---|
| msg319636 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2018-06-15 18:14 | |
The iov_setup() helper in posixmodule.c returns the total size of all buffers. But there is possible an integer overflow because the sequence of buffers can contain the same buffer repeated multiple times.On 32-bit platform:>>> import os>>> f = open('/tmp/temp', 'wb')>>> os.writev(f.fileno(), [b'x' * 2**16] * 2**15)-1Since the overflowed sum is negative, os_writev_impl() returns -1 as a signal of error, but since the exception is not set, -1 is returned as the result of os.writev(). If the overflowed sum is not negative, the sequence of buffers is passed to OS and an OSError is raised:>>> os.writev(f.fileno(), [b'x' * 2**16] * 2**16)Traceback (most recent call last): File "<stdin>", line 1, in <module>OSError: [Errno 22] Invalid argumentI have not tested (because have not installed corresponding 32-bit OSes, and it is harder to reproduce on 64-bit), but seems this can even cause a crash in os.sendfile() on FreeBSD, DragonFly BSD and Mac OS.This sum is used only in os.sendfile() on Mac OS. In all other cases it is enough to return just an error flag. I can't find the documentation for os.sendfile() on Mac OS for checking if this value actually is needed. | |||
| msg319641 -(view) | Author: Pablo Galindo Salgado (pablogsal)*![]() | Date: 2018-06-15 18:42 | |
I can reproduce the issue on a i686 GNU/Linux Debian system:>>> import os>>> f = open('/tmp/temp', 'wb')>>> l = os.writev(f.fileno(), [b'x' * 2**16] * 2**15)Traceback (most recent call last): File "<stdin>", line 1, in <module>SystemError: error return without exception set | |||
| msg319645 -(view) | Author: Pablo Galindo Salgado (pablogsal)*![]() | Date: 2018-06-15 18:47 | |
Docs for sendfile in macOS:https://www.unix.com/man-page/osx/2/sendfile/ | |||
| msg319757 -(view) | Author: Ned Deily (ned.deily)*![]() | Date: 2018-06-16 16:02 | |
Test case:import osfo = open('/tmp/temp', 'wb')fi = open('/tmp/temp', 'rb')os.sendfile(fo.fileno(), fi.fileno(), 0, 0, headers=[b'x' * 2**16] * 2**15)--run against current master HEAD (2f9cbaa8b2190b6dfd3157ede9b6973523a3b939, as of 2018-06-15)--with-pydebugcurrent macOS 10.13.564-bit Python$ ./bin/python3.8 ~/Desktop/test_s.pyTraceback (most recent call last): File "/Users/nad/Desktop/test_s.py", line 4, in <module> os.sendfile(fo.fileno(), fi.fileno(), 0, 0, headers=[b'x' * 2**16] * 2**15)OSError: [Errno 38] Socket operation on non-socketsys:1: ResourceWarning: unclosed file <_io.BufferedWriter name='/tmp/temp'>sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/temp'>32-bit Python$ ./bin/python3.8-32 ~/Desktop/test_s.pyFatal Python error: a function returned NULL without setting an errorSystemError: <built-in function sendfile> returned NULL without setting an errorCurrent thread 0xa983a1c0 (most recent call first): File "/Users/nad/Desktop/test_s.py", line 4 in <module>Abort trap: 6 | |||
| msg319879 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2018-06-18 12:56 | |
Thank you Pablo and Ned!Seems there is other bug on Mac OS, not related to integer overflow. I am working on it. | |||
| msg322739 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2018-07-31 07:24 | |
New changeset9d5727326af53ddd91016d98e16ae7cf829caa95 by Serhiy Storchaka in branch 'master':bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)https://github.com/python/cpython/commit/9d5727326af53ddd91016d98e16ae7cf829caa95 | |||
| msg322746 -(view) | Author: miss-islington (miss-islington) | Date: 2018-07-31 09:20 | |
New changeset3e4b68875917a4605b45918f9e3232730fed9399 by Miss Islington (bot) in branch '3.7':bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)https://github.com/python/cpython/commit/3e4b68875917a4605b45918f9e3232730fed9399 | |||
| msg322749 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2018-07-31 09:59 | |
New changesetada5d99306dc8af21c32cefb3d86891e8553dbc6 by Serhiy Storchaka in branch '3.6':[3.6]bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931) (GH-8584)https://github.com/python/cpython/commit/ada5d99306dc8af21c32cefb3d86891e8553dbc6 | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:01 | admin | set | github: 78052 |
| 2018-09-22 18:31:42 | serhiy.storchaka | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
| 2018-07-31 09:59:03 | serhiy.storchaka | set | messages: +msg322749 |
| 2018-07-31 09:20:13 | miss-islington | set | nosy: +miss-islington messages: +msg322746 |
| 2018-07-31 07:38:38 | serhiy.storchaka | set | pull_requests: +pull_request8093 |
| 2018-07-31 07:25:13 | miss-islington | set | pull_requests: +pull_request8092 |
| 2018-07-31 07:24:57 | serhiy.storchaka | set | messages: +msg322739 |
| 2018-06-26 16:46:56 | serhiy.storchaka | set | keywords: +patch stage: patch review pull_requests: +pull_request7538 |
| 2018-06-18 12:56:32 | serhiy.storchaka | set | priority: normal -> high messages: +msg319879 |
| 2018-06-16 16:02:48 | ned.deily | set | messages: +msg319757 |
| 2018-06-15 18:47:20 | pablogsal | set | messages: +msg319645 |
| 2018-06-15 18:42:22 | pablogsal | set | nosy: +pablogsal messages: +msg319641 |
| 2018-06-15 18:14:36 | serhiy.storchaka | create | |