Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
Description
Bug report
Steps:
- create a subinterpreter
- in it, start a subthread that takes one or more seconds to finish
- wait for that subthread to finish
One would expect execution to flow as follows:
| main interpreter main thread | subinterpreter main thread | subinterpreter subthread | |
|---|---|---|---|
| 1 | runtime init | ||
| 2 | run script | ||
| 3 | create subinterpreter | ||
| 4 | init | ||
| 5 | run code | ||
| 6 | start subthread | run code | |
| 7 | exit script | (run code) | |
| 8 | begin runtime fini | (run code) | |
| 9 | begin fini | (run code) | |
| 10 | wait for subthread | (run code) | |
| 11 | (wait for subthread) | done | |
| 12 | finish fini | ||
| 13 | finish fini | ||
| 14 | exit |
However, that isn't what's happening. Instead, at some point the subthread stops executing (without error) and the process effectively hangs (but can be stopped with Ctrl-C).
Script:
print('START: thread 0 (interp 0) - executing')importatexitimportthreadingfromtest.supportimportinterpretersdefnotify_before_thread_shutdown():print('START: thread 0 (interp 0) - waiting for non-daemon threads')threading._register_atexit(notify_before_thread_shutdown)defnotify_after_thread_shutdown():print('END: thread 0 (interp 0) - waiting for non-daemon threads')atexit.register(notify_after_thread_shutdown)interp=interpreters.create()deftask():print('START: thread 1 (interp 0) - executing')interp.exec_sync("""if True: print('START: thread 1 (interp 1) - executing') import atexit import threading def notify_before_thread_shutdown(): print('START: thread 0 (interp 1) - waiting for non-daemon threads') threading._register_atexit(notify_before_thread_shutdown) def notify_after_thread_shutdown(): print('END: thread 0 (interp 1) - waiting for non-daemon threads') atexit.register(notify_after_thread_shutdown) def task(): print('START: thread 2 (interp 1) - executing') import time for i in range(1, 11): if i == 6: print(' ', end='', flush=True) print('.', end='', flush=True) time.sleep(0.1) print(flush=True) print('END: thread 2 (interp 1) - executing') t = threading.Thread(target=task) t.start() # t.join() here makes the problem go away. print('END: thread 1 (interp 1) - executing') """)print('END: thread 1 (interp 0) - executing')t=threading.Thread(target=task)t.start()t.join()# interp.close() here makes the problem go away.# time.sleep(1) here makes the problem go away.print('END: thread 0 (interp 0) - executing')
Expected output:
START: thread 0 (interp 0) - executingSTART: thread 1 (interp 0) - executingSTART: thread 1 (interp 1) - executingSTART: thread 2 (interp 1) - executingEND: thread 1 (interp 1) - executingEND: thread 1 (interp 0) - executingEND: thread 0 (interp 0) - executingSTART: thread 0 (interp 0) - waiting for non-daemon threadsEND: thread 0 (interp 0) - waiting for non-daemon threadsSTART: thread 0 (interp 1) - waiting for non-daemon threads..... .....END: thread 2 (interp 1) - executingEND: thread 0 (interp 1) - waiting for non-daemon threadsExpected output:
START: thread 0 (interp 0) - executingSTART: thread 1 (interp 0) - executingSTART: thread 1 (interp 1) - executingSTART: thread 2 (interp 1) - executingEND: thread 1 (interp 1) - executingEND: thread 1 (interp 0) - executingEND: thread 0 (interp 0) - executingSTART: thread 0 (interp 0) - waiting for non-daemon threadsEND: thread 0 (interp 0) - waiting for non-daemon threadsSTART: thread 0 (interp 1) - waiting for non-daemon threadsI'm fairly sure I know what the problem is: our hack for dealing with daemon threads. Currently, between steps 8 and 9 above, we mark the runtime as finalizing. Any threads (regardless of interpreter) are more or less immediately killed.
However, as far as the threading module is concerned, they are still valid and running and will never stop. This isn't a problem for the main interpreter because at that point it already waited for it's own non-daemon threads to finish.
Thisis a problem for subinterpreters because they don't wait fortheir non-daemon threads to finish until later. Thus we end up waiting forever at step 10.
The solution? Immediately after waiting for non-daemon threads in step 8, finalize all subinterpreters. Alternately, we could make sure we're only killing daemon threads with our hack (e.g. don't use a global "finalizing" check).
Metadata
Metadata
Assignees
Labels
Projects
Status