This patch fixes a racing condition when using Pool.terminate()
My application is using imap() but aborting further processing after some iterations. Therefore I used terminate() to end the workers pool but the called join() of the result_handler was hanging sporadically.
I traced the issue down to the problem that somehow poll() signals new data in the outqueue but the preceding get() never returned due to ERROR_IO_PENDING in winapi of the queue hand endless hanging WaitForMultipleObjects().

Waiting at most for the two sentinels issued by the pool cleanup fixes the issue. Also the
while sentinel_count < 2 seems to me more reasonable and plattform-independent than the oldfor i in range(10).

https://bugs.python.org/issue33997

Fix racing condition in termination of pool result_handler

984057c

Fixes a racing condition when using Pool.terminate(). Sporadically the poll() function signals new data in the queue but get() never returns.This fix just consumes the two issued sentinels by the cleanup process.

Copy link

the-knights-who-say-ni commentedJun 29, 2018

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed thePSF contributor agreement (CLA).

Unfortunately we couldn't find an account corresponding to your GitHub username onbugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please followthe steps outlined in the CPython devguide to rectify this issue.

When your account is ready, please add a comment in this pull request
and a Python core developer will remove theCLA not signed label
to make the bot check again.

Thanks again for your contribution, we look forward to reviewing it!

the-knights-who-say-ni added the CLA not signed label

Jun 29, 2018

bedevere-bot added the awaiting review label

Jun 29, 2018

Copy link

Author

3mb3dw0rk5 commentedJun 29, 2018•
edited
Loading

Added missing "GitHub Name" entry to successful verify CLA.

Mariatta removed the CLA not signed label

Jul 13, 2018

the-knights-who-say-ni added the CLA signed label

Jul 13, 2018

remilapeyre reviewed

Aug 5, 2019

View reviewed changes

Copy link

Contributor

remilapeyre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks@3mb3dw0rk5

remilapeyre approved these changes

Aug 5, 2019

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels

Aug 5, 2019

pppery reviewed

Aug 5, 2019

View reviewed changes

Lib/multiprocessing/pool.py

		@@ -497,6 +499,7 @@ def _handle_results(outqueue, get, cache):

		if task is None:
		util.debug('result handler ignoring extra sentinel')
		sentinel_counter += 1