Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
Description
I went down this rabbit hole when someone mentioned thatisfile/isdir/exists all make a rather expensiveos.stat call on Windows (which is actually a long wrapper around a number of system calls on Windows), rather than the simpler and more direct call toGetFileAttributeW.
I noticed that at one point there was aversion ofisdir that does exactly this. At the time, this claimed a 2x speedup.
However, this C implementation ofisdir was removed as part of a large set of changes indf2d4a6, and as a result,isdir got faster.
With the following benchmark:
isdir benchmark
import os.pathimport timeitfor i in range(100): os.makedirs(f"exists{i}", exist_ok=True)def test_exists(): for i in range(100): os.path.isdir(f"exists{i}")def test_extinct(): for i in range(100): os.path.isdir(f"extinct{i}")print(timeit.timeit(test_exists, number=100))print(timeit.timeit(test_extinct, number=100))for i in range(100): os.rmdir(f"exists{i}")I get the following withdf2d4a6:
exists: 0.18694799999957468doesn't exist: 0.08418370000072173and with the prior commit:
exists: 0.25393609999991895doesn't exist: 0.08511730000009265So, from this, I'd conclude that the idea of replacing calls toos.stat with calls toGetFileAttributeW would not bear fruit, but@zooba should probably confirm I'm benchmarking the right thing and making sense.
In any event, we should probably remove thelittle vestige that imports this fast path that was removed:
try:# The genericpath.isdir implementation uses os.stat and checks the mode# attribute to tell whether or not the path is a directory.# This is overkill on Windows - just pass the path to GetFileAttributes# and check the attribute from there.fromntimport_isdirasisdirexceptImportError:# Use genericpath.isdir as imported above.pass