Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
Description
Bug report
Bug description:
This came up in#130849 (comment)
The problem is thatpopen_fork.Popen (andpopen_spawn.Popen andpopen_forkserver.Popen) are not thread-safe:
cpython/Lib/multiprocessing/popen_fork.py
Lines 25 to 35 in02de9cb
| defpoll(self,flag=os.WNOHANG): | |
| ifself.returncodeisNone: | |
| try: | |
| pid,sts=os.waitpid(self.pid,flag) | |
| exceptOSError: | |
| # Child process not yet created. See #1731717 | |
| # e.errno == errno.ECHILD == 10 | |
| returnNone | |
| ifpid==self.pid: | |
| self.returncode=os.waitstatus_to_exitcode(sts) | |
| returnself.returncode |
The first successful call toos.waitpid() may reap the pid so that subsequent calls raise anOSError. I've only seen this on macOS (not Linux). We may not yet however have setself.returncode -- that happens a few statements later, sopoll() can returnNone if:
- The process has finished
- Another thread called poll(), but hasn't yet set
self.returncode
And thenis_alive() can return True:
cpython/Lib/multiprocessing/process.py
Lines 153 to 170 in02de9cb
| defis_alive(self): | |
| ''' | |
| Return whether process is alive | |
| ''' | |
| self._check_closed() | |
| ifselfis_current_process: | |
| returnTrue | |
| assertself._parent_pid==os.getpid(),'can only test a child process' | |
| ifself._popenisNone: | |
| returnFalse | |
| returncode=self._popen.poll() | |
| ifreturncodeisNone: | |
| returnTrue | |
| else: | |
| _children.discard(self) | |
| returnFalse |
Note that some classes likeconcurrent.futures.ProcessPoolExecutor use threads internally, so the user may not even know that threads are involved.
Repro:
repro.py
importosimportmultiprocessingasmpimportthreadingimporttimeimportsysoriginal_excepthook=threading.excepthookdefon_except(args):original_excepthook(args)os._exit(1)threading.excepthook=on_exceptdefp1():passdefthread1(p):whilep.is_alive():time.sleep(0.00001)passdeftest():foriinrange(1000):print(i)p=mp.Process(target=p1)p.start()t=threading.Thread(target=thread1,args=(p,))t.start()p.join()assertnotp.is_alive()t.join()defmain():threads= [threading.Thread(target=test)for_inrange(10)]fortinthreads:t.start()fortinthreads:t.join()if__name__=="__main__":main()
NOTE:
- This is unrelated to free threading
popen_fork.Popen(and subclasses) are distinct fromsubprocess.Popen
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS