File date information can be Pita when used by file shares/zfs etc.

Option for sha256 or custom providers would allow sifferent kind of consistency checks.

Some filesystems allow to have hash build in and some have metadata/tags that allows this to be better for production.

For a love of devs, do not use regions 👏

This sync call will destroy some sharing services.
I suggest that try use it, if fails, then try create folder.

This is more ops per write than "normally" needed.

I mentioned this in discord, and wanted to note it here for anyone else looking at this PR. We did something similar and
getting the semantics for a file system storage provider is tricky because:

• File systems are implemented differently, and you have to be sure the subset of functionality you are using is robust across all of them - including when you add NFS or SMB into the mix too, what do they each guarantee in terms of data integrity?
• Writes are not atomic, if your program or OS crashes midway through overwriting grain state, you are left in a non-deterministic possibly corrupt state.
• LastWriteTimeUtc may not be accurate - caching, lazy metadata writes could affect it - seehttps://learn.microsoft.com/en-us/dotnet/api/system.io.filesysteminfo.lastwritetimeutc?view=net-9.0#remarks
• It's possible for two identical grains to be active during a split-brain scenario so you cannot rely on reading last write time then writing because that allows a race-condition.
• On Linux (as of .NET 9.0) there are no truly asynchronous file operations - all the async ones are implemented as synchronous queued on the threadpool, so you need to be careful not to flood the threadpool with thread stalling sync work during grain activation storage reads.

All these problems can be worked around, and I think it's important to do so because you are dealing with storage and people will trust it to reliably persist their precious data.

We went through several iterations for a log-based storage provider and we settled on was:

• use exclusive file locking (and we test this works on the base path specified on initialization, because if not all bets are off) - and handle the specific IOException HResult (which is different on Windows and *nix) trying to access a locked file
• append xxHash to the contents of each file to ensure we can detect partial writes/integrity problems - this could maybe also serve as your etag in this scenario
• always write replacement contents (with a xxHash) to a new deterministically named file and then overwrite the original file ensuring exclusive access to both for the duration.
• always look for the above deterministically named new file when opening each original file, and resume the replacement operation if it exists.
• Add a concurrency gate around the async-over-sync file operations and increase the threadpool size by the concurrency gate limit, to ensure the threadpool has adequate capacity for our file operations.
• Ensure file handle lifetime is short if the number of concurrent handles might become a problem (some storage stacks have limits)
• Make sure you are not going to run into inode exhaustion based on how you store your files especially for EXT volumes it seems.

With the above mitigated we have processed a few billion storage operations on the file system now - but we are still only using this for data that can be replaced.