Penny for your thoughts on this@encukou :)

This looks reasonable, butStatResult looks rather useless with its three mandatory but unreliable members:

• st_mode has UNIX permission bits and aplatform-specific representation of file type (since AFAIK, POSIX doesn't specify the values ofS_IFREG et.al.)
• st_ino &st_dev, which can be 0 if the info is not available -- if it weren't for the tuple API, the attributes could just as well be missing in this case.

Also, I guess typing has no way to say that e.g. ifst_birthtime is present, it needs to be a time in seconds.

It seems that for your goal of supporting virtual filesystems, it might be better to rely onDirEntry, withstat remaining as a backend for filesystems where it's useful.
I wonder if we need a FS-agnostic version ofst_ino/st_dev, like:

def key(self) -> Hashable:    """Return a comparison key. Entries with the same key refer to the same file."""    # For a Unix-like FS:    ...    return (stat_result.st_dev, stat_result.st_ino)

Thank you for the feedback and reviews.

It seems that for your goal of supporting virtual filesystems, it might be better to rely onDirEntry, withstat remaining as a backend for filesystems where it's useful. I wonder if we need a FS-agnostic version ofst_ino/st_dev, like:

def key(self) -> Hashable:    """Return a comparison key. Entries with the same key refer to the same file."""    # For a Unix-like FS:    ...    return (stat_result.st_dev, stat_result.st_ino)

I've occasionally thought about adding aPathBase.status() abstract method, which could return aDirEntry-like object with caching methods. Methods likePathBase.is_dir() could then be implemented viaself.status().is_dir(). If we did that, we could either deletePathBase.stat(), or implement it in terms ofself.status(). Would that be better?

I don't thinkDirEntry is an appropriate return type forstatus(), because we're not scanning the parent directory, rather we're querying the path directly. Perhaps it's just a naming problem.

I don't think DirEntry is an appropriate return type for status(), because we're not scanning the parent directory, rather we're querying the path directly. Perhaps it's just a naming problem.

Status, then? Likestat, but not a C-style abbr.
OrFileInfo?

I'm not a fan of thestat() model at all. It's the best of a bad lot, I know, but I'd rather have some kind of API that lets us use cached information foris_file()/is_dir()/etc. rather than having to guarantee that it's fresh for each call.

I'm not sure what such an API should look like, apart from I want it to have thePath object interface, and be using cached responses about the state of the path (or fetching them and caching on demand). There are more than enough properties that might as well be cached that it's worth it, and the TOCTOU issues basically all exist even with our current model.

Brainstorming:
Add aFileInfo protocol, implemented by classes that hold file info for a path. All the information in it is cached; the cache(s) might be initialized when an attribute is accessed (this canany attribute -- e.g. accessing the creation time might cache all ofstat()), or when theFileInfo is created. The cache(s) can't be cleared (externally,FileInfo is immutable; but you can make a fresh instance).
Paths would get acached_property-likeinfo attribute, initialized to aFileInfo either on first access or when the Path is created (e.g. to hold extra info from iterdir).
As withcached_property, the cache can be cleared usingdel path.info.
We'd need anew_info() method as well, to hand out fresh info but leave the cache alone -- mainly as a backend for the current Path methods.

A downside is that this breaks the rule that touching the filesystem is done by methods, not attributes. (Can we say rule was for getting “fresh” information, and we can use something else here?)

This all sounds great to me.

A downside is that this breaks the rule that touching the filesystem is done by methods, not attributes.

Why would accessingPath.info touch the filesystem? Couldn't make it so that onlyFileInfo.is_*() methods can perform FS access?

Oh, I suppose you might want tolstat() orstat() so thatFileInfo always represents something that exists/existed. Otherwise we'd need aFileInfo.exists() method, and if we're intending to storeos.DirEntry objects asPath.info when paths are generated fromiterdir(), then we'd need to add anos.DirEntry.exists() method too.

Courtesy CCing@ncoghlan, who proposed something similarhere. I shot it down then because it wasn't clear to me when to call[l]stat(), how to clear caches, and how it interacts withPath.is_*(). I'm also a tiny bit uneasy makingPath stateful.

(Process note - we should probably move the API design discussion off this PR)

exists() shouldn't be an issue, because we can cache that asTrue orFalse at creation time (e.g. returned fromglob() implies thatexists() == True at some point in the recent past; created by a user implies thatexists() is unknown and needs to make a filesystem call). The same could potentially be true for any other state, most obviouslyis_dir() andis_file(), but also any of the regular stat attributes (where a subclass could well return their own type from.stat()).

What we need to design is the way for a user to discover that they've got cached values, and reset it. ACachedInfo mixin (just for type check purposes) and a dedicated subclass (so that e.g.Path(p) would recreate and uncache all members) would seem to suffice, though I can also see the reasoning for adding a new member to manage it.

Following up on the forum:https://discuss.python.org/t/ergonomics-of-new-pathlib-path-scandir/71721/30

I've removedDirEntry andStatResult from this PR, so it now covers only onlyParser.

Thanks both, very much appreciate the feedback :-)