Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-135239: simpler use of mutex in hashlib & co#135267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
picnixz merged 45 commits intopython:mainfrompicnixz:perf/hashlib/mutex-135239
Jun 22, 2025
Merged
Changes from1 commit
Commits
Show all changes
45 commits
Select commitHold shift + click to select a range
5d8c093
add common object head for hashlib/hmac objects
picnixzJun 8, 2025
81e3046
simplify digest computation
picnixzJun 8, 2025
7f9f7b7
refactor update logic
picnixzJun 8, 2025
15a4f2f
refactor alloc() logic
picnixzJun 8, 2025
5cd828a
finalizing touches
picnixzJun 8, 2025
63db1de
correct mutex usage
picnixzJun 15, 2025
ea033a3
Revert 5cd828acdcfef753aee5eec7e13f07682af40f46
picnixzJun 15, 2025
77baa67
revert some constructor changes
picnixzJun 15, 2025
902759f
unconditionally lock when performing HASH updates
picnixzJun 16, 2025
dde68c4
Merge remote-tracking branch 'upstream/main' into perf/hashlib/mutex-…
picnixzJun 16, 2025
05c1e66
post-merge
picnixzJun 16, 2025
db57278
do not guard against empty buffers for now
picnixzJun 16, 2025
ead20a1
consistency fixes
picnixzJun 16, 2025
68a6bbc
remove unused import
picnixzJun 16, 2025
68f297e
correct naming for locked/unlocked versions
picnixzJun 16, 2025
9817c3d
debug?
picnixzJun 16, 2025
7c6842b
Merge remote-tracking branch 'upstream/main' into perf/hashlib/mutex-…
picnixzJun 16, 2025
c14c87d
simplify HMAC
picnixzJun 16, 2025
bfb5436
release the GIL for large buffers
picnixzJun 16, 2025
923c05f
restore GIL_MINSIZE
picnixzJun 16, 2025
55b2afa
correctly lock objects
picnixzJun 16, 2025
5cd60d1
improve tests
picnixzJun 16, 2025
a2fcbd5
fixup HMAC
picnixzJun 16, 2025
417cee1
fixup
picnixzJun 16, 2025
f350501
GIL protection
picnixzJun 16, 2025
5c4009d
show WASI errors
picnixzJun 16, 2025
8aec797
fix WASI
picnixzJun 16, 2025
6db58dc
fix compilation
picnixzJun 16, 2025
b1f9463
fix compilation
picnixzJun 16, 2025
491b922
fix warnings
picnixzJun 16, 2025
c048975
sync
picnixzJun 17, 2025
c9044d2
fixup format string
picnixzJun 17, 2025
6c08f0d
address review
picnixzJun 17, 2025
7fd1396
reudce diff
picnixzJun 20, 2025
f400a11
Merge remote-tracking branch 'upstream/main' into perf/hashlib/mutex-…
picnixzJun 20, 2025
5e2daa8
Merge remote-tracking branch 'upstream/main' into perf/hashlib/mutex-…
picnixzJun 20, 2025
4f9729e
Merge remote-tracking branch 'upstream/main' into perf/hashlib/mutex-…
picnixzJun 20, 2025
06aaee0
Merge branch 'main' into perf/hashlib/mutex-135239
picnixzJun 21, 2025
977c807
fixup
picnixzJun 21, 2025
6d66fef
fixup
picnixzJun 21, 2025
c9db0b1
make the test suite less slow
picnixzJun 21, 2025
6ffdd1c
fix test when GIL_MINSIZE is changed
picnixzJun 21, 2025
98ec915
defer cosmetics
picnixzJun 21, 2025
398ddb3
Update Lib/test/test_hashlib.py
picnixzJun 21, 2025
0ae70e9
improve test
picnixzJun 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
PrevPrevious commit
improve test
  • Loading branch information
@picnixz
picnixz committedJun 22, 2025
commit0ae70e9c3f3cac460a414ea8f9e77bbdd4c80748
33 changes: 16 additions & 17 deletionsLib/test/test_hashlib.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1055,7 +1055,7 @@ def test_sha256_gil(self):
@threading_helper.reap_threads
@threading_helper.requires_working_threading()
def test_threaded_hashing_fast(self):
# Same as test_threaded_hashing_slow() but only tests"fast" functions
# Same as test_threaded_hashing_slow() but only testssome functions
# since otherwise test_hashlib.py becomes too slow during development.
for name in ['md5', 'sha1', 'sha256', 'sha3_256', 'blake2s']:
if constructor := getattr(hashlib, name, None):
Expand All@@ -1081,30 +1081,29 @@ def do_test_threaded_hashing(self, constructor, is_shake):
# If the internal locks are working to prevent multiple
# updates on the same object from running at once, the resulting
# hash will be the same as doing it single threaded upfront.
#
# Be careful when choosing num_threads, len(smallest_data)
# and len(data) // len(smallest_data) as the obtained chunk
# size needs to satisfy some conditions below.
num_threads = 5
smallest_data = os.urandom(8)
data = smallest_data * 200000

# The data to hash has length s|M|q^N and the chunk size for the i-th
# thread is s|M|q^(N-i), where N is the number of threads, M is a fixed
# message of small length, and s >= 1 and q >= 2 are small integers.
smallest_size, num_threads, s, q = 8, 5, 2, 10

smallest_data = os.urandom(smallest_size)
data = s * smallest_data * (q ** num_threads)

h1 = constructor(usedforsecurity=False)
h2 = constructor(data * num_threads, usedforsecurity=False)

def hash_in_chunks(chunk_size):
index = 0
while index < len(data):
def update(chunk_size):
for index in range(0, len(data), chunk_size):
h1.update(data[index:index + chunk_size])
index += chunk_size

threads = []
for threadnum in range(num_threads):
chunk_size = len(data) // (10 ** threadnum)
for thread_num in range(num_threads):
# chunk_size = len(data) // (q ** thread_num)
chunk_size = s * smallest_size * q ** (num_threads - thread_num)
self.assertGreater(chunk_size, 0)
self.assertEqual(chunk_size % len(smallest_data), 0)
thread = threading.Thread(target=hash_in_chunks,
args=(chunk_size,))
self.assertEqual(chunk_size % smallest_size, 0)
thread = threading.Thread(target=update, args=(chunk_size,))
threads.append(thread)

for thread in threads:
Expand Down
Loading

[8]ページ先頭

©2009-2025 Movatter.jp