Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.2k
bpo-33997: Fix racing condition in termination of pool result_handler#8009
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Changes from1 commit
File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
Fixes a racing condition when using Pool.terminate(). Sporadically the poll() function signals new data in the queue but get() never returns.This fix just consumes the two issued sentinels by the cleanup process.
- Loading branch information
Uh oh!
There was an error while loading.Please reload this page.
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -464,6 +464,7 @@ def _handle_tasks(taskqueue, put, outqueue, pool, cache): | ||
@staticmethod | ||
def _handle_results(outqueue, get, cache): | ||
thread = threading.current_thread() | ||
sentinel_counter = 0 | ||
while 1: | ||
try: | ||
@@ -479,6 +480,7 @@ def _handle_results(outqueue, get, cache): | ||
if task is None: | ||
util.debug('result handler got sentinel') | ||
sentinel_counter += 1 | ||
break | ||
job, i, obj = task | ||
@@ -497,6 +499,7 @@ def _handle_results(outqueue, get, cache): | ||
if task is None: | ||
util.debug('result handler ignoring extra sentinel') | ||
sentinel_counter += 1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. This extra sentinel is no longer ignored, so the debug is wrong. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. Obviously right. Fixed that. | ||
continue | ||
job, i, obj = task | ||
try: | ||
@@ -511,10 +514,12 @@ def _handle_results(outqueue, get, cache): | ||
# attempts to add the sentinel (None) to outqueue may | ||
# block. There is guaranteed to be no more than 2 sentinels. | ||
try: | ||
while sentinel_counter < 2: | ||
if not outqueue._reader.poll(): | ||
break | ||
task = get() | ||
if task is None: | ||
sentinel_counter += 1 | ||
except (OSError, EOFError): | ||
pass | ||