Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-111545: Add Py_HashPointer() function#112096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
vstinner merged 3 commits intopython:mainfromvstinner:hash_pointer
Dec 6, 2023

Conversation

vstinner
Copy link
Member

@vstinnervstinner commentedNov 15, 2023
edited
Loading

  • Keep _Py_HashPointer() function as an alias to PyHash_Pointer().
  • Add tests on PyHash_Pointer() to test_capi.test_hash.
  • Remove _Py_HashPointerRaw() function: inline code in _Py_hashtable_hash_ptr().

📚 Documentation preview 📚:https://cpython-previews--112096.org.readthedocs.build/

@vstinner
Copy link
MemberAuthor

I checked how_Py_rotateright_uintptr() is compiled on x86-64:

  • clang -O3 with__builtin_rotateright64():ror rax, cl
  • gcc -O3 with(x >> bits) | (x << (8 * SIZEOF_UINTPTR_T - bits)):ror rbx,cl

As expected, theROR instruction is used in both cases.


I usedgdb: in Python, I runimport _testinternalcapi, and then in gdb I typedisassemble rotateright_uintptr.

For that, I made the following change. Thecheck_rotateright_uintptr() cannot be used to check the machine code: sincecheck_rotateright_uintptr() arguments are constant, GCC and clang just pre-compute the function result at build. The funciton is no longer called at runtime.

Change:

diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.cindex 2d888e0ae2c..a08b3dfb168 100644--- a/Modules/_testinternalcapi.c+++ b/Modules/_testinternalcapi.c@@ -290,6 +290,23 @@ test_rotateright_uintptr(PyObject *self, PyObject *Py_UNUSED(args)) }+static PyObject*+rotateright_uintptr(PyObject *self, PyObject *args)+{+    PyObject *obj;+    int bits;+    if (!PyArg_ParseTuple(args, "O!i", &PyLong_Type, &obj, &bits)) {+        return NULL;+    }+    unsigned long long x = PyLong_AsUnsignedLongLong(obj);+    if (x == (unsigned long long)-1 && PyErr_Occurred()) {+        return NULL;+    }+    uintptr_t y = _Py_rotateright_uintptr((uintptr_t)x, bits);+    return PyLong_FromUnsignedLongLong(y);+}++ #define TO_PTR(ch) ((void*)(uintptr_t)ch) #define FROM_PTR(ptr) ((uintptr_t)ptr) #define VALUE(key) (1 + ((int)(key) - 'a'))@@ -1689,6 +1706,7 @@ static PyMethodDef module_functions[] = {     {"test_popcount", test_popcount, METH_NOARGS},     {"test_bit_length", test_bit_length, METH_NOARGS},     {"test_rotateright_uintptr", test_rotateright_uintptr, METH_NOARGS},+    {"rotateright_uintptr", rotateright_uintptr, METH_VARARGS},     {"test_hashtable", test_hashtable, METH_NOARGS},     {"get_config", test_get_config, METH_NOARGS},     {"set_config", test_set_config, METH_O},

@vstinnervstinner marked this pull request as ready for reviewNovember 15, 2023 11:12
@vstinnervstinner requested review froma team andtiran ascode ownersNovember 15, 2023 11:12
@vstinnervstinnerforce-pushed thehash_pointer branch 4 times, most recently from992c39e tof63b726CompareNovember 15, 2023 11:33
@vstinner
Copy link
MemberAuthor

vstinner commentedNov 15, 2023
edited
Loading

I checked how _Py_rotateright_uintptr() is compiled on x86-64

Adding_Py_rotateright_uintptr() static inline function added too much code, and it's not needed: current code is already compiled to the expected most efficient x86-64ROR instruction. I removed the_Py_rotateright_uintptr() function.

@vstinner
Copy link
MemberAuthor

Changes:

@vstinner
Copy link
MemberAuthor

@encukou: Please review the updated PR.

@vstinnervstinner changed the titlegh-111545: Add PyHash_Pointer() functiongh-111545: Add Py_HashPointer() functionDec 1, 2023
Copy link
Member

@encukouencukou left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This looks great to me, thank you for adding it!

I hope Serhiy is OK with this iteration. (And that the rest of the C-API WG won't have major objections, but Idon't think that's likely.)

@vstinner
Copy link
MemberAuthor

@serhiy-storchaka: Do you want to review this change?

@vstinner
Copy link
MemberAuthor

I rebased the PR on main to get a NoGIL fix.

@vstinner
Copy link
MemberAuthor

@encukou:

(And that the rest of the C-API WG won't have major objections, but Idon't think that's likely.)

PEP 731 says:

The working group may operate primarily through reviews of GitHub issues and PRs.

Are you suggesting that the C API cannot be modified anymore unless whole C API Working Group approve a change? Or are you suggesting that the C API Working Group must review all C API additions? I don't think that was the plan. Well, the plan is not well defined.

@encukou
Copy link
Member

encukou commentedDec 4, 2023
edited
Loading

Are you suggesting that the C API cannot be modified anymore unless whole C API Working Group approve a change?

No, I'm saying that the others might disagree -- and that I don't think it's very likely that they will disagree.
I'd optimistically merge this (edit: after checking with Serhiy), but be prepared to fix it up or revert.

Copy link
Member

@zoobazooba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No concerns from me, other than the lack of docs.

Adding new C API should include the WG up until we've developed and agreed upon the guidelines for adding new C API. Without those, yeah, we should all get pinged for any new APIs. (And if that takes up all our time and we don't get a chance to develop guidelines, so be it...)


.. c:function:: Py_hash_t Py_HashPointer(const void *ptr)

Hash a pointer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can we add a bit more text to clarify that it ensures the pointer itself is usable as a hash value, and does not even attempt to access the memory the pointer refers to (let alone trying to call__hash__ or similar)?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I completed the doc. Is it more explicit? I clarified that only the pointervalue is hashed.

* Implement _Py_HashPointerRaw() as a static inline function.* Add Py_HashPointer() tests to test_capi.test_hash.* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
@vstinner
Copy link
MemberAuthor

Adding new C API should include the WG up until we've developed and agreed upon the guidelines for adding new C API.

@encukou createdcapi-workgroup/decisions#1

``uintptr_t`` internally). The pointer is not dereferenced.

The function cannot fail: it cannot return ``-1``.

Copy link
Member

@iritkatrieliritkatrielDec 6, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'm missing some mention of the use case for this (not necessarily here). The issue talks about_Py_HashDouble, there is no mention ofPy_HashPointer.

@vstinner
Copy link
MemberAuthor

vstinner commentedDec 6, 2023
edited
Loading

@iritkatriel:

I'm missing some mention of the use case for this (not necessarily here). The issue talks about _Py_HashDouble, there is no mention of Py_HashPointer.

ProposedPy_HashDouble() API requires the caller to callPy_HashPointer(obj) for NaN: see#112449

Moreover, I removed the private function_Py_HashPointer() in Python 3.13 alpha1 and it broke a few projects. A code search on PyPI top 5,000 projects found 4 projects:

  • cffi (1.16.0)
  • gmpy2 (2.1.5)
  • numba (0.58.1)
  • tree_sitter (0.20.4)

Finally, igraph-0.11.2 contains a copy of the private_Py_HashPointer() function insrc/_igraph/pyhelpers.c.

@vstinnervstinner merged commit828451d intopython:mainDec 6, 2023
@vstinnervstinner deleted the hash_pointer branchDecember 6, 2023 14:09
@vstinner
Copy link
MemberAuthor

I merged my PR. The API was approved by the C API Working Group:capi-workgroup/decisions#1

aisk pushed a commit to aisk/cpython that referenced this pull requestFeb 11, 2024
* Implement _Py_HashPointerRaw() as a static inline function.* Add Py_HashPointer() tests to test_capi.test_hash.* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
Glyphack pushed a commit to Glyphack/cpython that referenced this pull requestSep 2, 2024
* Implement _Py_HashPointerRaw() as a static inline function.* Add Py_HashPointer() tests to test_capi.test_hash.* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@iritkatrieliritkatrieliritkatriel left review comments

@zoobazoobazooba left review comments

@encukouencukouencukou approved these changes

@tirantiranAwaiting requested review from tirantiran is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@vstinner@encukou@iritkatriel@zooba

[8]ページ先頭

©2009-2025 Movatter.jp