Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

bpo-34395: Don't free allocated memory on realloc fail in load_mark() in _pickle.c.#8788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
serhiy-storchaka merged 3 commits intopython:masterfromsir-sigurd:bpo-34395
Aug 25, 2018

Conversation

@sir-sigurd
Copy link
Contributor

@sir-sigurdsir-sigurd commentedAug 16, 2018•
edited by bedevere-bot
Loading

alloc <= ((size_t)self->num_marks+1)) {
PyErr_NoMemory();
return-1;
}
Copy link
Member

@serhiy-storchakaserhiy-storchakaAug 16, 2018•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Seems the check above can be removed.alloc > (PY_SSIZE_T_MAX / sizeof(Py_ssize_t)) is duplicated inPyMem_RESIZE(), and failingalloc <= ((size_t)self->num_marks + 1) is not possible ifsizeof(Py_ssize_t) > 2.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

if ((self->num_marks+1) >=self->marks_size) {

Also, it seems that new space is needed only whenself->num_marks + 1 is strictly greater thanself->marks_size.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Agree. This condition can be written asself->num_marks >= self->marks_size orself->num_marks == self->marks_size.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@serhiy-storchaka
I'd prefer to make these change as separate commits/RPs as they are not related to this bpo issue.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

PR for upsizing issue#8860.

@zhangyangyu
Copy link
Member

Need backport to make code consistent or as enhancement?

@serhiy-storchaka
Copy link
Member

I don't think there is a bug that needs to be fixed. This change looks rather as a code clean up to me. If I'm wrong, and it fixes a real bug, it should be backported.

@sir-sigurd, please remove also the redundant check before PyMem_RESIZE if it is truly unneeded.

@sir-sigurd
Copy link
ContributorAuthor

@serhiy-storchaka

I'm considering adding this as explanation why overflow check is not needed

assert((size_t)self->num_marks <= PY_SSIZE_T_MAX / sizeof(Py_ssize_t));Py_BUILD_ASSERT((size_t)PY_SSIZE_T_MAX / sizeof(Py_ssize_t) << 1 <= SIZE_MAX - 20);

but I'm not sure if it makes things clear.

@serhiy-storchaka
Copy link
Member

I think this is not needed. Just remove the check.

@serhiy-storchakaserhiy-storchaka merged commit90555ec intopython:masterAug 25, 2018
@sir-sigurdsir-sigurd deleted the bpo-34395 branchAugust 25, 2018 10:42
@zhangyangyu
Copy link
Member

zhangyangyu commentedAug 25, 2018•
edited
Loading

failing alloc <= ((size_t)self->num_marks + 1) is not possible if sizeof(Py_ssize_t) > 2.

@serhiy-storchaka Can this be guaranteed in CPython? Wikipedia says size_t is at least 16 bit wise.

CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull requestAug 27, 2018
* master: (104 commits)  Fast path for exact floats in math.hypot() and math.dist() (pythonGH-8949)  Remove AIX workaround test_subprocess (pythonGH-8939)  bpo-34503: Fix refleak in PyErr_SetObject() (pythonGH-8934)  closes bpo-34504: Remove the useless NULL check in PySequence_Check(). (pythonGH-8935)  closes bpo-34501: PyType_FromSpecWithBases: Check spec->name before dereferencing it. (pythonGH-8930)  closes bpo-34502: Remove a note about utf8_mode from sys.exit() docs. (pythonGH-8928)  Remove unneeded PyErr_Clear() in _winapi_SetNamedPipeHandleState_impl() (pythonGH-8281)  Fix markup in stdtypes documentation (pythonGH-8905)  bpo-34395: Don't free allocated memory on realloc fail in load_mark() in _pickle.c. (pythonGH-8788)  Fix upsizing of marks stack in pickle module. (pythonGH-8860)  bpo-34171: Prevent creating Lib/trace.cover when run the trace module. (pythonGH-8841)  closes bpo-34493: Objects/genobject.c: Add missing NULL check to compute_cr_origin() (pythonGH-8911)  Fixed typo with asynccontextmanager code example (pythonGH-8845)  bpo-34426: fix typo (__lltrace__ -> __ltrace__) (pythonGH-8822)  bpo-13312: Avoid int underflow in time year. (pythonGH-8912)  bpo-34492: Python/coreconfig.c: Fix _Py_wstrlist_copy() (pythonGH-8910)  bpo-34448: Improve output of usable wchar_t check (pythonGH-8846)  closes bpo-34471: _datetime: Add missing NULL check to tzinfo_from_isoformat_results. (pythonGH-8869)  bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks (pythonGH-8864)  Fix typo in the dataclasses's doc (pythonGH-8896)  ...
@serhiy-storchaka
Copy link
Member

I think we can assume thatsize_t is at least 32 bit wise, andchar is 8 bit wise. Many things in CPython implementation will be broken if this is not true.

@sir-sigurd
Copy link
ContributorAuthor

If I'm not mistaken, it can't overflow even if sizeof(Py_ssize_t) == 2 (presuming that CPython allocates at most PY_SSIZE_T_MAX bytes).

@zhangyangyu
Copy link
Member

If I'm not mistaken, it can't overflow even if sizeof(Py_ssize_t) == 2 (presuming that CPython allocates at most PY_SSIZE_T_MAX bytes).

I don't get it, how it's not possible.

And rechecking the patch it seems to me the overflow check is not right. You should check overflow before doing math operation.size_t alloc = ((size_t)self->num_marks << 1) + 20; might already overflowed and succeed in the overflow check inPyMem_Resize. What do you think about it? Am I mistaken something?

@sir-sigurd
Copy link
ContributorAuthor

@zhangyangyu
Presuming that CPython allocates at mostPY_SSIZE_T_MAX bytes, we can assume that(size_t)self->num_marks <= PY_SSIZE_T_MAX / sizeof(Py_ssize_t). So
alloc <= (PY_SSIZE_T_MAX / sizeof(Py_ssize_t) << 1) + 20. Ifsizeof(Py_ssize_t) == 2, thenalloc <= PY_SSIZE_T_MAX + 20 and it's obviously less thanSIZE_MAX.

Makes sense?

@zhangyangyu
Copy link
Member

@sir-sigurd Good insight! Thanks for the explanation.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@serhiy-storchakaserhiy-storchakaserhiy-storchaka approved these changes

@zhangyangyuzhangyangyuzhangyangyu approved these changes

Assignees

No one assigned

Labels

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sir-sigurd@zhangyangyu@serhiy-storchaka@the-knights-who-say-ni@bedevere-bot
[8]ページ先頭
©2009-2025 Movatter.jp