What's wrong with Windows?

I still have only a cursory understanding (or recollection...) of this part of asyncio, and I worry that we're applying "programming by random modification". How confident are you that you understand what goes on here, rather than just observing the tests pass?

This is the version I am most confident with. I wrote asummary about this with and without fix which would hopefully help you in reviewing. I have spent weeks on trying to get the best fix for this and this is the best I could come up with.

There are so many win32 checks in the test... Maybe it would be better to have two versions of the test, guarded by @skipIf platform Win or not?

Regarding windows, if you see the tests in this file, you'll see that there are literally no tests testing killing of subprocess withshell=True i.e.asyncio.create_subprocess_shell. This is the first test which tests it, I won't be surprised if these tests were intentionally omitted because no-one wanted deal with subprocess shell on Windows. (From my experience shell and console is worst on Windows).

Thanks@kumaraditya303 for the PR, and@gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

GH-97915 is a backport of this pull request to the3.11 branch.

GH-97916 is a backport of this pull request to the3.10 branch.

AMD64 Ubuntu Shared 3.x buildbot started to fail recently with Out-of-Memory (OOM) errors: Python test processes are killed with SIGKILL by the Linux kernel.

David Bolen, owner of the buildbot worker, wrote that there's currently an awful lot (340) of processes like:(...)/python -c import time; time.sleep(100000000). I guess that these processes come fromtest_kill_issue43884() of test_asyncio.test_subprocess (Lib/test/test_asyncio/test_subprocess.py). He added: "Looks like they started accumulating sporadically on Oct 2".

Maybe this change supposed to kill asyncio processes more cleanly is now making the situation worse? Or maybe it's a different regression?

I ran the following command in 3 terminals on Linux to stress test the test:

./python -m test -m test_kill_issue43884 -j40 test_asyncio test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio  test_asyncio

While tests are running, I see Python processes runningtime.sleep(100000000). But once tests complete, I don't see these processes anymore.

Maybe the issue was fixed in the meanwhile? I don't know.

So, there are already a new set of hung processes on my buildbot since I killed everything a short while ago. There's an active 3.11 build going on, but the asyncio test already completed, so it sounds like those processes should be gone. There are two clusters (3 processes) about 16 minutes apart, so I'm guessing one from the last build and one from this one. test_asyncio did run near the end of the last build and near the beginning of the current one.

As soon as I have some free cycles, I'll try some manual tests on the buildbot to see if I can reproduce. But in the meantime I'll monitor to ensure they don't accumulate enough to interfere again with subsequent builds.

I created issue#98407: "AMD64 Ubuntu Shared 3.x: python processes killed with SIGKILL by Linux Out-of-Memory (OOM), maybe related to test_asyncio".