Repository files navigation

zipref is a script that generates (uncompressed) ZIP files on copy-on-write filesystems by ref-linking the contents of the source files inside the archive.

It does so by (ab)using the fact that the ZIP format allows padding between entries to carefully align the file header for each entry so that the data starts at an offset which is a multiple of block size. This allows cloning the data from the source file directly.

The resulting archive is a proper ZIP64 archive and should barely take up any extra space. It has been tested to extract properly in WinRAR,7z andunzip.

Note that the process is not instantaneous: the data still has to be read in order to compute the CRC32 for each entry. Additionally, if the cloneioctl fails, the script will revert to copying the data manually.

The script takes the name of the output archive as first argument, as well as the name of files to add as additional arguments. It's possible to use@ to specify a file containing a list of files, and to use- to read said list from standard input.

The output file must not exist already, and the input files are addedas-is in the specified order, using their paths verbatim as they are specified.

Some examples:

# Specify file list on the command linezipref output.zip*.mp3# Specify file list from an input filezipref output.zip @files.txt# Specify file list from standard inputfind -type f| zipref output.zip

Additionally, it is possible to generate a reproducible archive using theSOURCE_DATE_EPOCH environment variable:

# Generate reproducible archiveexport TZ=UTCexport SOURCE_DATE_EPOCH=1600000000# or $(stat -c %Y reference_file)find -type f| sort -f| zipref output.zip

• ZIP64 - Go Big Or Go Home