- Notifications
You must be signed in to change notification settings - Fork6
Another Virtual File System for Go
License
avfs/avfs
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Another Virtual File System for Go
AVFS is a virtual file system abstraction, inspired mostlybyAfero and Go standard library. It providesan abstraction layer to emulate the behavior of a file system that provides several features :
- a set ofconstants,interfaces andtypes for all file systems
- atest suite for all file systems (emulated or real)
- each file system has itsown package
- a very basicidentity manager allows testing of user related functions(Chown, Lchown) and file system permissions
Additionally, some file systems support :
- user file creation mode mask (Umask) (MemFS, OrefaFS)
- chroot (OSFS on Linux)
- hard links (MemFS, OrefaFS)
- symbolic links (MemFS)
- multiple users concurrently (MemFS)
- Linux andWindows emulation regardless of host operating system (MemFS, OrefaFS)
This package can be installed with the go install command :
go install github.com/avfs/avfs@latest
It is only tested with Go version >= 1.18
To make an existing code work withAVFS :
- replace all references of
os
,path/filepath
with the variable used to initialize the file system (vfs
in the followingexamples) - import the packages of the file systems and, if necessary, the
avfs
packageand initialize the file system variable. - some file systems provide specific options available at initialization. Forinstance
MemFS
needsWithSystemDirs
option to create/home
,/root
and/tmp
directories.
The example below demonstrates the creation of a file, a symbolic link to thisfile, for a different file systems (depending on an environment variable). Errormanagement has been omitted for the sake of simplicity :
package mainimport ("bytes""log""os""github.com/avfs/avfs""github.com/avfs/avfs/vfs/memfs""github.com/avfs/avfs/vfs/osfs")funcmain() {varvfs avfs.VFSswitchos.Getenv("ENV") {case"PROD":// The real file system for production.vfs=osfs.NewWithNoIdm()default:// in memory for tests.vfs=memfs.New()}// From this point all references of 'os', 'path/filepath'// should be replaced by 'vfs'rootDir,_:=vfs.MkdirTemp("","avfs")defervfs.RemoveAll(rootDir)aFilePath:=vfs.Join(rootDir,"aFile.txt")content:= []byte("randomContent")_=vfs.WriteFile(aFilePath,content,0o644)aFilePathSl:=vfs.Join(rootDir,"aFileSymlink.txt")_=vfs.Symlink(aFilePath,aFilePathSl)gotContentSl,_:=vfs.ReadFile(aFilePathSl)if!bytes.Equal(content,gotContentSl) {log.Fatalf("Symlink %s : want content to be %v, got %v",aFilePathSl,content,gotContentSl)}log.Printf("content from symbolic link %s : %s",aFilePathSl,gotContentSl)}
The example below demonstrates the concurrent creation of subdirectories under aroot directory by several users in different goroutines (works only withMemFS) :
package mainimport ("fmt""log""sync""github.com/avfs/avfs""github.com/avfs/avfs/idm/memidm""github.com/avfs/avfs/vfs/memfs")funcmain() {const (maxUsers=100groupName="test_users")idm:=memidm.New()vfs:=memfs.New()rootDir,_:=vfs.MkdirTemp("","avfs")vfs.Chmod(rootDir,0o777)g,_:=idm.GroupAdd(groupName)varwg sync.WaitGroupwg.Add(maxUsers)fori:=0;i<maxUsers;i++ {gofunc(iint) {deferwg.Done()userName:=fmt.Sprintf("user_%08d",i)idm.UserAdd(userName,g.Name())vfsU,_:=vfs.Sub("/")vfsU.SetUser(userName)path:=vfsU.Join(rootDir,userName)vfsU.Mkdir(path,avfs.DefaultDirPerm)}(i)}wg.Wait()entries,_:=vfs.ReadDir(rootDir)log.Println("number of dirs :",len(entries))for_,entry:=rangeentries {info,_:=entry.Info()sst:=vfs.ToSysStat(info)u,_:=idm.LookupUserId(sst.Uid())log.Println("dir :",info.Name(),", mode :",info.Mode(),", owner :",u.Name())}}
Almost ready for Windows.
All file systems implement at leastavfs.FS
andavfs.File
interfaces.By default, each file system supported methods are the most commonly used frompackagesos
andpath/filepath
. All methods have identical names as theirfunctions counterparts.The following file systems are currently available :
File system | Comments |
---|---|
BasePathFS | file system that restricts all operations to a given path within a file system |
MemFS | In memory file system supporting major features of a linux file system (hard links, symbolic links, chroot, umask) |
OrefaFS | Afero like in memory file system |
OsFS | Operating system native file system |
RoFS | Read only file system |
File system methodsavfs.VFS | Comments |
---|---|
Abs | equivalent tofilepath.Abs |
Base | equivalent tofilepath.Base |
Chdir | equivalent toos.Chdir |
Chmod | equivalent toos.Chmod |
Chown | equivalent toos.Chown |
Chtimes | equivalent toos.Chtimes |
Clean | equivalent tofilepath.Clean |
Create | equivalent toos.Create |
CreateTemp | equivalent toos.CreateTemp |
Dir | equivalent tofilepath.Dir |
EvalSymlinks | equivalent tofilepath.EvalSymlinks |
FromSlash | equivalent tofilepath.FromSlash |
Features | returns the set of features provided by the file system or identity manager |
Getwd | equivalent toos.Getwd |
Glob | equivalent tofilepath.Glob |
HasFeature | returns true if the file system or identity manager provides a given feature |
Idm | returns the identity manager of the file system |
IsAbs | equivalent tofilepath.IsAbs |
IsPathSeparator | equivalent tofilepath.IsPathSeparator |
Join | equivalent tofilepath.Join |
Lchown | equivalent toos.Lchown |
Link | equivalent toos.Link |
Lstat | equivalent toos.Lstat |
Match | equivalent tofilepath.Match |
Mkdir | equivalent toos.Mkdir |
MkdirAll | equivalent toos.MkdirAll |
MkdirTemp | equivalent toos.MkdirTemp |
Open | equivalent toos.Open |
OpenFile | equivalent toos.OpenFile |
OSType | returns the operating system type of the file system |
PathSeparator | equivalent toos.PathSeparator |
ReadDir | equivalent toos.ReadDir |
ReadFile | equivalent toos.ReadFile |
Readlink | equivalent toos.Readlink |
Rel | equivalent tofilepath.Rel |
Remove | equivalent toos.Remove |
RemoveAll | equivalent toos.RemoveAll |
Rename | equivalent toos.Rename |
SameFile | equivalent toos.SameFile |
SetUMask | sets the file mode creation mask |
SetUser | sets and returns the current user |
Split | equivalent tofilepath.Split |
Stat | equivalent toos.Stat |
Sub | equivalent tofs.Sub |
Symlink | equivalent toos.Symlink |
TempDir | equivalent toos.TempDir |
ToSlash | equivalent tofilepath.ToSlash |
ToSysStat | takes a value from fs.FileInfo.Sys() and returns a value that implements interface avfs.SysStater |
Truncate | equivalent toos.Truncate |
UMask | returns the file mode creation mask |
User | returns the current user |
Utils | returns the file utils of the current file system |
WalkDir | equivalent tofilepath.WalkDir |
WriteFile | equivalent toos.WriteFile |
File methodsavfs.File | Comments |
---|---|
Chdir | equivalent toos.File.Chdir |
Chmod | equivalent toos.File.Chmod |
Chown | equivalent toos.File.Chown |
Close | equivalent toos.File.Close |
Fd | equivalent toos.File.Fd |
Name | equivalent toos.File.Name |
Read | equivalent toos.File.Read |
ReadAt | equivalent toos.File.ReadAt |
ReadDir | equivalent toos.File.ReadDir |
Readdirnames | equivalent toos.File.Readdirnames |
Seek | equivalent toos.File.Seek |
Stat | equivalent toos.File.Stat |
Truncate | equivalent toos.File.Truncate |
Write | equivalent toos.File.Write |
WriteAt | equivalent toos.File.WriteAt |
WriteString | equivalent toos.File.WriteString |
Identity managers allow users and groups management. The ones implementedinavfs
are just here to allow testing of functions related to users (Chown,Lchown) and access rights, so they just allow one default group per user.
All file systems supporting identity manager implement by default the identitymanagerDummyIdm
where all functions returnsavfs.ErrPermDenied
.
Identity Manager | Comments |
---|---|
DummyIdm | dummy identity manager where all functions are not implemented |
MemIdm | In memory identity manager |
OsIdm | Identity manager using os functions |
SQLiteIdm | Identity manager backed by a SQLite database |
Identity Manager methodsavfs.FS avfs.IdentityMgr | Comments |
---|---|
AdminGroup | returns the administrator group (root for Linux) |
AdminUser | returns the administrator user (root for Linux) |
GroupAdd | adds a new group |
GroupDel | deletes an existing group |
LookupGroup | looks up a group by name |
LookupGroupId | looks up a group by groupid |
LookupUser | looks up a user by username |
LookupUserId | looks up a user by userid |
UserAdd | adds a new user |
UserDel | deletes an existing user |
About
Another Virtual File System for Go