@brettcannon a long time ago ina PR far, far away, you said:

My current thinking (while I work through my PR review backlog to reach this PR), is this should be used forzipfile.Path. I also wouldn't be opposed to that happening this PR if@barneygale wanted to give that a shot while he waits for a review from me.

I couldn't make this work forzipfile.Path due to a couple of fundamental differences in the interface, e.g. theroot attribute stores a string inpathlib.PurePath, but aZipFile instance inzipfile.Path. It's possible to make it work with a couple of deprecations/removals, or an entirely newZipPath class.

So in this PR I've chosentarfile.TarPath as the initial target. I think it does a good job of showing the viability of thepathlib._VirtualPath interface.

However, I have a bunch of open questions in the PR description about howTarPath itself should work. It's not as straightforward as.zip files, because tarballs support symlinks and hardlinks, and becausetarfile.TarFile doesn't support reading and writing the same file.

These questions make me think thattarfile.TarPath should first be gestated in a PyPI package before we add it to the standard library. But that leaves a bit of a chicken-and-egg problem with gettingpathlib._VirtualPath in.

Do you have any advice on the best way forward?

Thanks!

These questions make me think thattarfile.TarPath should first be gestated in a PyPI package before we add it to the standard library. But that leaves a bit of a chicken-and-egg problem with gettingpathlib._VirtualPath in.

You have a couple of options:

1. Start a discussion on discuss.python.org fortarfile and see if there's clear consensus on what to do.
2. Make_VirtualPath as private and do your experimental PyPI project knowing it might be perpetually tied to a specific Python version (and make that clear to users of the package).
3. Go with your gut and hope for the best (and err on the side of leaving things out, especially if they can be added in later). 😅

I am unsure about the name VirtualPath!

The way I’ve seen it used, a TarPath class, or SshPath, or S3Path would be examples of virtual filesystems: I give them paths that look like regular filesystem paths and they do custom operations to return info or data.

Their base class is not itself a virtual path, it’s an (abstract?) base class!

Fair point! How aboutPathBase? It squares with the names of base classes inio, I think.

Alternatively,pathlib.abc.VirtualPath?

Not a fan of deep modules, and even.abc.VirtualPath doesn’t seem great to me.
I have posted on discuss about VirtualPath vs PathBase!

just stumbled uppon this and im wondering if a concept of a public Accessor of a path might make sense
where an accessor provides the implementation, then for example tarfile/zipfile paths could just have the accessor

additionally it would be trivial to have "roots" at a path or a "root" with a directory file descriptor/network protocol

just stumbled uppon this and im wondering if a concept of a public Accessor of a path might make sense where an accessor provides the implementation, then for example tarfile/zipfile paths could just have the accessor

We actually used to have accessor objects, but their interface was much the same asPath objects, and so they amounted to nothing more than a layer of indirection. They were removed in#25701.

If we add accessors back in, most users would need to subclasstwo classes to implement their own path objects: the accessor class and the path class. The latter would be necessary for any user who wants to add their own path methods, for example.

those "accessors" where nothing more than aliases, after all the implementation as done had no power over what the pure path portion can do with the real world

a actualy "accessor" would be able to be tied to something like a directory file descriptor, or a path object, or a entirely different thing thats not a filesystem

and yes there is some need for synchronization between concrete path capabilities and the accessors,

however there is certain equivalence classes to keep in mind (Posix, Windows), so in a lot of cases my impression is that there will be more "accessors" than paths

Intriguing. Could you sketch out the accessor API?

(in the forums rather than a PR discussion please)

i can dedicate some time to it later

a basic concept of the accessor version ought to consider something like:

• windows paths

• posix paths

• unc paths

• directory file descriptor paths (both secured and unsecured)

• paths that pretend to be a chroot on top of another path/pathlike

• archives

• ssh/scp

• urls/http

• distinctions between readonly and readwrite accessors

• capabilities of accessors (like users/perms/xattrs)

• wrappers (like union fs) to support interesting testing
  (imagine a path that maps onto the real fs but pretends certain files are deleted/replaced)

  additionally for purely virtual accessors it might be interesting if parts of the path could be objects (like dates), but thats currently out of scope

To summarise my thoughts on the above: in many cases this PR helps bring those things about, and I don't think it hinders any of those things. But let's chat more in a forum thread.

What's left in this PR:

• LandGH-109187: Improve symlink loop handling inpathlib.Path.resolve() #109192 to align handling of symlink loops inPath.resolve()'s with this implementation of_PathBase.resolve(), and withos.path.realpath()
• Get review/approval for this_PathBase.resolve() implementation.

On the latter point, there are still two subtle and related problems:

• _PathBase.resolve(strict=True) can raiseOSError with the wrong values (errno, path). For example,resolve('/etc/hosts/foo') should raiseNotADirectoryError(path='/etc/hosts'), but instead raisesFileNotFoundError(path='/etc/hosts/foo')
• To avoid an infinite loop, the implementation of_PathBase.readlink() must not callself.resolve(), and so users must supply a path with no symlinks, except in the final position. This is a bit different toPath.readlink().

Both of these issues could be solved if we can provide some way tostat() a path without resolvingany symlinks. Theresolve() method would call thisstat() method first, and only callreadlink() if the stat result indicates a symlink. A couple of options spring to mind:

• We could allow_PathBase.stat(follow_symlinks=None), meaning "where possible, don't followany symlinks in the path; the path I've provided is the direct physical path". But it's not particularly obvious, andfollow_symlinks=None means something different inglob() andrglob().
• We could add some other keyword-only argument to_PathBase.stat() with the same effect.
• We could add a new method --direct_stat(),stat1(),dstat()?

Bah, pushed a commit labelled "WIP". In fact it contains a solution to the above problem: stash a_resolving flag on path objects, which meansresolve() can safely callstat() andreadlink() without hitting infinite recursion. No weird APIs for users.

Hey@AA-Turner and@gst, thank you for your earlier reviews, very much appreciated. Please could you give it another pass, if you can spare the time?

I feel OK about_PathBase.resolve(): it's entirely private for the timebeing, and I intend to add further tests and optimization before we drop the underscore from_PathBase, so if we can't spot any clear defects I suspect it's fine to land? Grateful for your guidance!

Thank you so much for the reviews, everyone :-)

Congratulations Barney, a lot of hard work on your part paid off!

A