Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-105235: Prevent reading outside buffer during mmap.find()#105252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
sweeneyde merged 13 commits intopython:mainfromsweeneyde:mmap_find_overflow
Jul 13, 2023

Conversation

@sweeneyde
Copy link
Member

@sweeneydesweeneyde commentedJun 2, 2023•
edited by bedevere-bot
Loading

@sweeneydesweeneyde changed the titlePrevent reading outside buffer during mmap.find()gh-105235: Prevent reading outside buffer during mmap.find()Jun 2, 2023
@sweeneydesweeneyde marked this pull request as ready for reviewJune 2, 2023 20:14
@sweeneydesweeneyde requested a review fromYhg1sJune 2, 2023 20:15
@sweeneydesweeneyde added needs backport to 3.11only security fixes needs backport to 3.12only security fixes labelsJun 2, 2023
@bacher09
Copy link

Hey, just want to add my two cents. Instead of usingfm.find(b"fo") you can usefm.find(b"fo", -2), since there is no point in going over the whole region, just doing the last match is enough, and it makes this whole test much faster: 0.1 seconds vs 17 seconds on my machine. And IMHO, it's nice to replace4096 withmmap.PAGESIZE, document that0 is aPROT_NONE andfd argument should be-1 and not0, it was my typo.

There is also another option is to useMAP_FIXED for making guard pages, which would allow avoiding doing loop entirely, but since python'smmap isn't allowing doing this, and doing this viactypes is extremely hacky I wouldn't recommend doing this, unless this would be done systematically inside the interpreter, or perhaps with something likethis.
Anyway,0.1 of second is probably a good enough for a test.

ambv and sweeneyde reacted with thumbs up emoji

@sweeneydesweeneyde added the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJun 8, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@sweeneyde for commit8b4a3d2 🤖

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJun 8, 2023
@sweeneydesweeneyde added the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJul 11, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@sweeneyde for commit1ac84ce 🤖

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJul 11, 2023
@sweeneyde
Copy link
MemberAuthor

A summary of buildbot failures:

AMD64 Arch Linux TraceRefs PR    test.test__xxsubinterpreters.RunStringTests.test_SystemExit    _Py_ForgetReference: Assertion failed: invalid object chainAMD64 Fedora Stable Clang Installed PRPPC64LE Fedora Stable Clang Installed PRaarch64 Fedora Stable Clang Installed PRs390x Fedora Clang Installed PRx86 Gentoo Installed with X PR    test.test_venv.BasicTest.test_zippath_from_non_installed_posix        No such file or directory: '/tmp/tmp1czueb9b/bin/python3'    test.test_venv.BasicTest.test_upgrade_dependencies        '/tmp/tmp0xz8yht5/bin/python3' != '/tmp/tmp0xz8yht5/bin/python3.13'AMD64 Fedora Stable Refleaks PRAMD64 RHEL7 Refleaks PRAMD64 RHEL8 Refleaks PRPPC64LE Fedora Stable Refleaks PRPPC64LE RHEL7 Refleaks PRPPC64LE RHEL8 Refleaks PRaarch64 Fedora Stable Refleaks PRaarch64 RHEL8 Refleaks PR    test.test_capi.test_misc.TestUops.test_extended_arg    <uop_executor object at 0x21b64d0> is not NoneAMD64 Windows11 Bigmem PR    test_peg_generator failed (env changed)AMD64 Windows11 Refleaks PR    test_import leaked [41, 43, 40] references, sum=124    test_import leaked [41, 42, 40] memory blocks, sum=123    also same EXTENDED_ARG failure as abovePPC64 Fedora PRs390x Debian PRs390x Fedora Clang PRs390x RHEL8 PRs390x RHEL8 Refleaks PRs390x SLES PR    python: ../Python/instrumentation.c:262: _PyInstruction_GetLength: Assertion `opcode != 0' failed.PPC64LE Fedora Stable Clang PR    test.test_gdb.PyBtTests.test_pycfunction    '<built-in method meth_varargs' not found in 'Breakpoint 1 (meth_varargs) ...'s390x Fedora LTO + PGO PR    test_tools.test_freeze_simple_script timeout after 15:00s390x RHEL8 LTO + PGO PR    test_tools.test_i18n.test_POT_Creation_Date timeout after 15:00s390x RHEL7 LTO + PGO PR    compile error    find: ‘build’: No such file or directory    also Python/instrumentation.c:1419:9: warning: missing braces around initializer [-Wmissing-braces]s390x RHEL7 LTO PRs390x RHEL7 PRs390x RHEL7 Refleaks PR    No space left on device    Also warnings:        Objects/unicodeobject.c:12898:23: warning: ‘subobj’ may be used uninitialized in this function [-Wmaybe-uninitialized]        Objects/unicodeobject.c:12844:23: warning: ‘subobj’ may be used uninitialized in this function [-Wmaybe-uninitialized]

@sweeneydesweeneyde merged commitab86426 intopython:mainJul 13, 2023
@miss-islington
Copy link
Contributor

Thanks@sweeneyde for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@sweeneydesweeneyde deleted the mmap_find_overflow branchJuly 13, 2023 02:50
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJul 13, 2023
…ythonGH-105252)* Add a special case for s[-m:] == p in _PyBytes_Find* Add tests for _PyBytes_Find* Make sure that start <= end in mmap.find(cherry picked from commitab86426)Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
@bedevere-bot
Copy link

GH-106708 is a backport of this pull request to the3.12 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.12only security fixes labelJul 13, 2023
@miss-islington
Copy link
Contributor

Sorry,@sweeneyde, I could not cleanly backport this to3.11 due to a conflict.
Please backport usingcherry_picker on command line.
cherry_picker ab86426a3472ab68747815299d390b213793c3d1 3.11

@bedevere-bot
Copy link

GH-106710 is a backport of this pull request to the3.11 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.11only security fixes labelJul 13, 2023
sweeneyde added a commit that referenced this pull requestJul 15, 2023
…H-105252) (#106708)gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)* Add a special case for s[-m:] == p in _PyBytes_Find* Add tests for _PyBytes_Find* Make sure that start <= end in mmap.find(cherry picked from commitab86426)Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
sweeneyde added a commit that referenced this pull requestJul 15, 2023
#106710)[3.11]gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)* Add a special case for s[-m:] == p in _PyBytes_Find* Add tests for _PyBytes_Find* Make sure that start <= end in mmap.find.(cherry picked from commitab86426)
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 Arch Linux TraceRefs 3.12 has failed when building commit4f3edd6.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1197/builds/135) and take a look at the build logs.
  4. Check if the failure is related to this commit (4f3edd6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1197/builds/135

Failed tests:

  • test_capi

Summary of the results of the build (if available):

==

Click to see traceback logs
remote:Enumerating objects: 15, done.remote:Counting objects:   7% (1/13)remote:Counting objects:  15% (2/13)remote:Counting objects:  23% (3/13)remote:Counting objects:  30% (4/13)remote:Counting objects:  38% (5/13)remote:Counting objects:  46% (6/13)remote:Counting objects:  53% (7/13)remote:Counting objects:  61% (8/13)remote:Counting objects:  69% (9/13)remote:Counting objects:  76% (10/13)remote:Counting objects:  84% (11/13)remote:Counting objects:  92% (12/13)remote:Counting objects: 100% (13/13)remote:Counting objects: 100% (13/13), done.remote:Compressing objects:  20% (1/5)remote:Compressing objects:  40% (2/5)remote:Compressing objects:  60% (3/5)remote:Compressing objects:  80% (4/5)remote:Compressing objects: 100% (5/5)remote:Compressing objects: 100% (5/5), done.remote:Total 15 (delta 8), reused 11 (delta 8), pack-reused 2From https://github.com/python/cpython * branch                  3.12       -> FETCH_HEADNote:switching to '4f3edd6b535b6a0b7352df134c0f445ab279bfc0'.You are in 'detached HEAD' state. You can look around, make experimentalchanges and commit them, and you can discard any commits you make in thisstate without impacting any branches by switching back to a branch.If you want to create a new branch to retain commits you create, you maydo so (now or later) by using -c with the switch command. Example:  git switch -c <new-branch-name>Or undo this operation with:  git switch -Turn off this advice by setting config variable advice.detachedHead to falseHEAD is now at 4f3edd6b53 [3.12] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252) (#106708)Switched to and reset branch '3.12'Objects/object.c:2212: _Py_ForgetReference: Assertion failed: invalid object chainEnable tracemalloc to get the memory block allocation tracebackobject address  : 0x7f06c47429d0object refcount : 0object type     : 0x5575f46d88c0object type name: bytesobject repr     : <refcnt 0 at 0x7f06c47429d0>Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailedPython runtime state: initializedCurrent thread 0x00007f06d67d8740 (most recent call first):  <no Python frame>Extension modules: _testinternalcapi (total: 1)Debug memory block at address p=0x7f06d6701e60: API '�'18302063728033390045bytes originally requested    The7 padbytes at p-7 arenotall FORBIDDENBYTE (0xfd):        at p-7:0xdd***OUCH        at p-6:0xdd***OUCH        at p-5:0xdd***OUCH        at p-4:0xdd***OUCH        at p-3:0xdd***OUCH        at p-2:0xdd***OUCH        at p-1:0xdd***OUCH    Because memoryis corrupted at the start, the count ofbytes requested       may be bogus,and checking the trailing padbytes may segfault.    The8 padbytes at tail=0xfdfe7d04d46dfc3d are Fatal Python error: Segmentation faultCurrent thread 0x00007f06d67d8740 (most recent call first):  <no Python frame>Extension modules: _testcapi, _testmultiphase, _testsinglephase, _xxsubinterpreters, _testinternalcapi (total: 5)make:*** [Makefile:2015: buildbottest] Segmentation fault (core dumped)Cannot open file '/buildbot/buildarea/3.12.pablogsal-arch-x86_64/build/test-results.xml' for upload

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@Yhg1sYhg1sAwaiting requested review from Yhg1s

Assignees

@sweeneydesweeneyde

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sweeneyde@bacher09@bedevere-bot@miss-islington
[8]ページ先頭
©2009-2025 Movatter.jp