- Notifications
You must be signed in to change notification settings - Fork2
A blazing fast concurrent zip archiver and extractor.
License
ybirader/pzip
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
pzip, short for parallel-zip, is a blazing fast concurrent zip archiver and extractor.
- Archives files and directories into a valid zip archive, using DEFLATE.
- Preserves modification times of files.
- Files are read and compressed concurrently
For command-line usage, we provide two binaries which can be installed separately:
- pzip- concurrent zip archiving
- punzip- concurrent zip extraction
To install, run:
For zip archiving:brew install ybirader/pzip/pzip
For zip extraction:brew install ybirader/pzip/punzip
For the latest stable release:
curl -1sLf 'https://dl.cloudsmith.io/public/pzip/stable/setup.deb.sh' | sudo -E bashsudo apt updatesudo apt install pzip
curl -1sLf 'https://dl.cloudsmith.io/public/pzip/stable/setup.deb.sh' | sudo -E bashsudo apt updatesudo apt install punzip
Alternatively, if you have Go installed:
go install github.com/ybirader/pzip
To build from source, we require Go 1.21 or newer.
- Clone the repository by running
git clone "https://github.com/ybirader/pzip.git"
- Build both pzip and punzip by running
make build
or build separately viacd cmd/pzip && go build
andcd cmd/punzip && go build
pzip
's API is similar to that of the standard zip utlity found on most *-nix systems.
pzip /path/to/compressed.zip path/to/file_or_directory1 path/to/file_or_directory2 ... path/to/file_or_directoryN
Alternatively, pzip can be imported as a package
archive,err:=os.Create("archive.zip")iferr!=nil {log.Fatal(err)}archiver,err:=pzip.NewArchiver(archive)iferr!=nil {log.Fatal(err)}deferarchiver.Close()files:= []string{"./hello","./hello.txt","./bye.md" }err=archiver.Archive(context.Background(),files)iferr!=nil {log.Fatal(err)}
The concurrency of the archiver can be configured using the corresponding flag:
pzip --concurrency 2 /path/to/compressed.zip path/to/file_or_directory1 path/to/file_or_directory2 ... path/to/file_or_directoryN
or by passing theArchiverConcurrency
option:
archiver,err:=pzip.NewArchiver(archive,ArchiverConcurrency(2))
punzip
's API is similar to that of the standard unzip utlity found on most *-nix systems.
punzip /path/to/compressed.zip
By default,punzip
extracts into the current directory. We can extract to a particular path by:
punzip -d /path/to/output /path/to/compressed.zip
Using the Go package, we have:
outputDirPath:="./output"archivePath:="./archive.zip"extractor,err:=pzip.NewExtractor(outputDirPath)iferr!=nil {log.Fatal(err)}deferextractor.Close()err=extractor.Extract(context.Background(),archivePath)iferr!=nil {log.Fatal(err)}
As with pzip, we can configure the concurrency of the extractor using:
punzip --concurrency 2 /path/to/compressed.zip
Similarly, with the Go package, we pass in theExtractorConcurrency
option:
extractor,err:=pzip.NewExtractor(outputDirPath,ExtractorConcurrency(2))
pzip was benchmarked using Matt Mahoney'ssample directory.
Using the standardzip
utlity, we get the following time to archive:
real 14m31.809suser 13m12.833ssys 0m24.193s
Running the same benchmark with pzip, we find that:
real 0m56.851suser 3m32.619ssys 1m25.040s
To contribute to pzip, first submit or comment in an issue to discuss your contribution, then open a pull request (PR).
pzip is released under theApache 2.0 license.
Many thanks to the folks atCloudsmith for graciously providing Debian package hosting. Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence.
About
A blazing fast concurrent zip archiver and extractor.