Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.3k
Closed
Description
Bug report
Bug description:
Below code demonstrates two problems that occur when
(a) worker process throws an exception, and
(b)shutdown(wait=False)
is used
- When max_workers < number of submitted tasks, raises
exception TypeError: object of type 'NoneType' has no len()
- When max_workers > number of submitted tasks, raises
exception TypeError: object of type 'NoneType' has no len()
AND then program hangs
There is no problem whenshutdown(wait=True)
is used or if the worker processes exit cleanly.
importconcurrent.futuresimportmultiprocessingasmpimporttimedeftask(n:int)->int:ifn==2:raiseException("Not gonna do it")else:time.sleep(n)returnndefmain()->None:# max_workers=2 results in raised exception and program exits# max_workers=4 results in raised exception and program hangs#executor=concurrent.futures.ProcessPoolExecutor(max_workers=2,mp_context=mp.get_context("forkserver"),max_tasks_per_child=1 )f1=executor.submit(task,1)f2=executor.submit(task,2)f3=executor.submit(task,3)result=0try:result+=f1.result()result+=f2.result()result+=f3.result()print(f"Result ={result}")exceptExceptionase:print(f"Exception while getting result :{e}")executor.shutdown(wait=False)if__name__=="__main__":main()
CPython versions tested on:
3.12
Operating systems tested on:
Linux, macOS
Linked PRs
- [3.12] gh-132969: Fix exception/hang shutdown(wait=False) and a task exited abnormally #133220
- gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally #133222
- [3.13] gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222) #135343
- [3.14] gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222) #135344
- gh-132969: ACKS file update was missed in gh-133222 #136144
Metadata
Metadata
Assignees
Labels
Projects
Status
Done