Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

xl-storage: improve xlStorage.openFile performance#21702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
charlievieth wants to merge1 commit intominio:master
base:master
Choose a base branch
Loading
fromcharlievieth:cev/xl-storage-openfile-perf
Open
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
xl-storage: improve xlStorage.openFile performance
This commit improves the performance of xlStorage.openFile and thusAppendFile by ~33% when the file's parent directory already exists.The performance improvement comes from changing openFile tooptimistically attempt to open the file and only create the parentdirectories if os.OpenFile fails with ErrNotExist.The performance impact of a failed call to os.OpenFile when the parentdirectory does not exist and must be created is about ~300ns on Linux,which is far less than the benefit this change provides for the likelymore common case of the parent directory existing.Benchmark results:goos: linuxgoarch: arm64pkg: github.com/minio/minio/cmd                      │ base.10.txt │             new.10.txt              │                      │   sec/op    │   sec/op     vs base                │XLStorageAppendFile-4   9.023µ ± 1%   5.961µ ± 0%  -33.94% (p=0.000 n=10)                      │ base.10.txt  │              new.10.txt              │                      │     B/op     │     B/op      vs base                │XLStorageAppendFile-4   2.281Ki ± 1%   1.672Ki ± 0%  -26.71% (p=0.000 n=10)                      │ base.10.txt │             new.10.txt             │                      │  allocs/op  │ allocs/op   vs base                │XLStorageAppendFile-4    50.00 ± 0%   35.00 ± 0%  -30.00% (p=0.000 n=10)Signed-off-by: Charlie Vieth <charlie.vieth@gmail.com>
  • Loading branch information
@charlievieth
charlievieth committedNov 20, 2025
commit15e4f8d444d906f6a70b37d992c47e47e4d8cf0e
14 changes: 9 additions & 5 deletionscmd/xl-storage.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1980,13 +1980,17 @@ func (s *xlStorage) openFile(filePath string, mode int, skipParent string) (f *o
if skipParent == "" {
skipParent = s.drivePath
}
// Create top level directories if they don't exist.
// with mode 0777 mkdir honors system umask.
if err = mkdirAll(pathutil.Dir(filePath), 0o777, skipParent); err != nil {
return nil, osErrToFileErr(err)
}

w, err := OpenFile(filePath, mode, 0o666)
if err != nil && os.IsNotExist(err) {
// Write likely failed because the parent directory does not
// exist. Attempt to create top-level directories with mode
// 0777 and retry the write.
if e1 := mkdirAll(pathutil.Dir(filePath), 0o777, skipParent); e1 != nil {
return nil, osErrToFileErr(e1)
}
w, err = OpenFile(filePath, mode, 0o666)
}
if err != nil {
// File path cannot be verified since one of the parents is a file.
switch {
Expand Down
31 changes: 31 additions & 0 deletionscmd/xl-storage_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1941,3 +1941,34 @@ func TestXLStorageReadMetadata(t *testing.T) {
t.Fatalf("Unexpected error from readMetadata - expect %v: got %v", errFileNameTooLong, err)
}
}

// TestXLStorage xlStorage.AppendFile() - benchmark performance of a zero length write.
// This benchmark is mostly designed to bench the performance of all of the setup code
// performed around the AppendFile and openFile methods.
func BenchmarkXLStorageAppendFile(b *testing.B) {
// create xlStorage
xlStorage, path, err := newXLStorageTestSetup(b)
if err != nil {
b.Fatalf("Unable to create xlStorage test setup, %s", err)
}

// Setup environment.
if err = xlStorage.MakeVol(b.Context(), "success-vol"); err != nil {
b.Fatalf("Unable to create volume, %s", err)
}

// Create directory to make errIsNotRegular
if err = os.Mkdir(slashpath.Join(path, "success-vol", "object-as-dir"), 0o777); err != nil {
b.Fatalf("Unable to create directory, %s", err)
}
b.ResetTimer()

fileName := "path/to/my/object"
data := []byte{}
for i := 0; i < b.N; i++ {
err := xlStorage.AppendFile(b.Context(), "success-vol", fileName, data)
if err != nil {
b.Fatal(err)
}
}
}
Loading

[8]ページ先頭

©2009-2025 Movatter.jp