Are you planning to also fix this in 3.10 and 3.11? That requires fixing_from_parts().

>>> pathlib.WindowsPath._from_parts(['C:\\spam', 'C:eggs'])WindowsPath('C:eggs')

Maybe the fix for_from_parts() should be a separate PR.

Are you planning to also fix this in 3.10 and 3.11? That requires fixing_from_parts().

>>> pathlib.WindowsPath._from_parts(['C:\\spam', 'C:eggs'])WindowsPath('C:eggs')

Maybe the fix for_from_parts() should be a separate PR.

I fixed this in#95450 and we decided not to backport because it was seemed like quite an obscure bug. I'd like to do the same here as the bug seems equally obscure (there are no user-logged bugs on the tracker that I could find), but I don't mind backporting both if we collectively feel that it's important.

With fresher eyes I realise my tests don't exercise per-drive working directories.

One way to address this would be to add asubst() context manager toLib/test/support/os_helper.py and use it fromtest_absolute(). The context manager would callsubst to allocate a new drive letter on__enter__(), andsubst /d on__exit__(). But perhaps there's a better way,@eryksun?

I fixed this in#95450 and we decided not to backport because it was seemed like quite an obscure bug.

Maybe skip fixing this in 3.10, butabsolute() is a documented method in 3.11. It should be able to resolve relative drive paths such as "Z:spam", which date back to MS-DOS in the 1980s. Maybe there's a more surgical fix for handling relative drive paths in_from_parts() that doesn't require usingntpath.join().

Maybe, but the bug is observable just fromPath('C:\\foo') / 'C:', and constructors have been documented the entire time. I don't think I can make the fix for#94909 much more surgical than#95450 because pathlib's original algorithm for joining segments (right-to-left scanning) doesn't lend itself well to the problem. Any fix I attemped resembledos.path.join().

I don't think the presence of this bug in 3.11 makesabsolute() so defective as to not warrant documentation.

Does that seem reasonable?

With fresher eyes I realise my tests don't exercise per-drive working directories.

BASE isos.path.realpath(TESTFN), which is in the initial current working directory. The_getfullpathname() branch is tested as long as this resolves to a real path on a drive-letter drive.

One way to address this would be to add asubst() context manager toLib/test/support/os_helper.py and use it fromtest_absolute(). The context manager would callsubst

The "subst.exe" app callsDefineDosDeviceW() without the flagDDD_NO_BROADCAST_SYSTEM. Without this flag, aWM_DEVICECHANGE message for eitherDBT_DEVICEARRIVAL orDBT_DEVICEREMOVECOMPLETE is broadcasted to the top-level windows of applications in the current session viaBroadCastSystemMessageW().

I wouldn't want to announce the new drive in the current session (i.e.DBT_DEVICEARRIVAL), particularly not to the desktop shell, Explorer¹. Instead,QueryDosDeviceW() andDefineDosDeviceW() could be added to_winapi, or called usingctypes. For example:

try:importctypesexceptImportError:defsubst_drive(path):raiseNotImplementedError('ctypes is not available')else:importcontextlibimportstringERROR_FILE_NOT_FOUND=2DDD_REMOVE_DEFINITION=2DDD_EXACT_MATCH_ON_REMOVE=4DDD_NO_BROADCAST_SYSTEM=8kernel32=ctypes.WinDLL('kernel32',use_last_error=True)@contextlib.contextmanagerdefsubst_drive(path):"""Temporarily yield a substitute drive for a given path."""forcinreversed(string.ascii_uppercase):drive=f'{c}:'if (notkernel32.QueryDosDeviceW(drive,None,0)andctypes.get_last_error()==ERROR_FILE_NOT_FOUND):breakelse:raiseFileExistsError('no available logical drive')ifnotkernel32.DefineDosDeviceW(DDD_NO_BROADCAST_SYSTEM,drive,path):raisectypes.WinError(ctypes.get_last_error())try:yielddrivefinally:ifnotkernel32.DefineDosDeviceW(DDD_REMOVE_DEFINITION|DDD_EXACT_MATCH_ON_REMOVE,drive,path):raisectypes.WinError(ctypes.get_last_error())

I wouldn't want to announce the new drive in the current session (i.e. DBT_DEVICEARRIVAL), particularly not to the desktop shell, Explorer

Could you expand on why this is a bad idea please? Thank you!

Could you expand on why this is a bad idea please?

This is a drive that we're creating for our own temporary use. I don't think it's appropriate to have Explorer windows automatically updated to include it. I also wouldn't want unknown complications from what any other application might do when sent aWM_DEVICECHANGE message forDBT_DEVICEARRIVAL.

Also, it's more efficient to directly callQueryDosDeviceW() in a loop when searching for an available drive than it is to repeatedly spawn a "subst.exe" process.

Gotcha, thanks.

Is there a time-of-check to time-of-use issue with your code above? I wonder if we could skip theQueryDosDeviceW checks and instead callDefineDosDeviceW with different drive letters until it succeeds?

Is there a time-of-check to time-of-use issue with your code above? I wonder if we could skip theQueryDosDeviceW checks and instead callDefineDosDeviceW with different drive letters until it succeeds?

I don't know how to avoid the small race condition without resorting to the NT API. "subst.exe" doesn't handle this any better.

DefineDosDeviceW() tries to be clever. The device names are defined as symbolic link objects in NT, e.g. "\??\Z:" -> "\??\C:\Windows". The target path of the symbolic link is a counted string with a current length and a maximum length, like most strings in NT.DefineDosDeviceW() uses this to implement a scheme that allows defining a device multiple times. The most recent definition is the active one, and each definition can be individually removed from the target buffer. For example:

>>> flags=DDD_NO_BROADCAST_SYSTEM>>> kernel32.DefineDosDeviceW(flags,'Z:','C:\\Windows')1>>> kernel32.DefineDosDeviceW(flags,'Z:','C:\\ProgramData')1>>> kernel32.DefineDosDeviceW(flags,'Z:','C:\\Temp')1>>> os.path.samefile('Z:','C:\\Temp')True>>> flags=DDD_REMOVE_DEFINITION|DDD_EXACT_MATCH_ON_REMOVE>>> kernel32.DefineDosDeviceW(flags,'Z:','C:\\Temp')1>>> os.path.samefile('Z:','C:\\ProgramData')True>>> kernel32.DefineDosDeviceW(flags,'Z:','C:\\ProgramData')1>>> os.path.samefile('Z:','C:\\Windows')True

I think this is bad behavior. The new drive definition affects all processes in the current logon session (i.e. the local device context). I wouldn't want to mask an existing drive definition. At least redefining a system drive requires administrator access.

Thanks@barneygale for the PR, and@zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

Thanks@barneygale for the PR, and@zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

Sorry,@barneygale and@zooba, I could not cleanly backport this to3.10 due to a conflict.
Please backport usingcherry_picker on command line.
cherry_picker 072011b3c38f871cdc3ab62630ea2234d09456d1 3.10

Sorry@barneygale and@zooba, I had trouble checking out the3.11 backport branch.
Please retry by removing and re-adding the "needs backport to 3.11" label.
Alternatively, you can backport usingcherry_picker on the command line.
cherry_picker 072011b3c38f871cdc3ab62630ea2234d09456d1 3.11

Thanks@barneygale for the PR, and@zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

Sorry,@barneygale and@zooba, I could not cleanly backport this to3.11 due to a conflict.
Please backport usingcherry_picker on command line.
cherry_picker 072011b3c38f871cdc3ab62630ea2234d09456d1 3.11

@barneygale Can I leave the backports with you?

GH-101993 is a backport of this pull request to the3.11 branch.

@zooba looks like this won't backport unless we also backport#95450. OK to skip?

Yeah, skip it.