With the attached script, then demo() called with for exampledatasize=40*1024*1024 and timeout=1 will deadlock: the program neverterminates.The bug appears on Linux (RHEL4) / intel x86 with "multiprocessing"coming with python 2.6b3 and I think it can be easily reproduced onother Unices. It also appears with python 2.5 and the standaloneprocessing package 0.52(https://developer.berlios.de/bugs/?func=detailbug&bug_id=14453&group_id=9001).After a quick investigation, it seems to be a deadlock between waitpidin the parent process, and a pipe::send in the "_feed" thread of thechild process. Indeed, the problem seems to be that "_feed" is stillsending data (the data is laaarge) to the pipe while the parent processalready called waitpid (because of the "short" timeout): the pipe fillsup because no consumer is eating the data (consumer already in waitpid)and hence the "_feed" thread in the child blocks forever. Since thechild process does a _feed.join() before exiting (after function f), itnever exits. And hence the waitpid in the parent process never returnsbecause the child never exits.This doesn't happen anymore if I use timeout=None or a larger timeout(eg. 10 seconds). Because in both cases, waitpid is called /after/ the"_feed" thread in the child process could send all of its data throughthe pipe.

A quick fix in the user code, when we are sure we don't need the childprocess if a timeout happens, is to call worker.terminate() in an exceptEmpty clause.

Seehttp://docs.python.org/dev/library/multiprocessing.html#multiprocessing-programmingSpecifically:Joining processes that use queuesBear in mind that a process that has put items in a queue will wait before terminating until all the buffered items are fed by the “feeder” thread to the underlying pipe. (The child process can call the Queue.cancel_join() method of the queue to avoid this behaviour.)This means that whenever you use a queue you need to make sure that all items which have been put on the queue will eventually be removed before the process is joined. Otherwise you cannot be sure that processes which have put items on the queue will terminate. Remember also that non-daemonic processes will be automatically be joined.

In a later release, I'd like to massage this in such a way that you do not have to wait for a child queue to be drained prior to calling join.One way to work around this David, is to call Queue.cancel_join_thread():def f(datasize, q):    q.cancel_join_thread()    q.put(range(datasize))

Thank you Jesse. When I read this passage, I thought naively that atimeout raised in a get() would not be harmful: that somehow the wholeget() request would be aborted. But now I realize that it would makethings rather complicated and dangerous: the data would get dropped, andwill never be recovered by subsequent get().So thank you for the hint, and leave the things as they are, it's better.

No problem David, you're the 4th person to ask me about this in the past 2 months :)