Repository files navigation

(C) Martin Väth (martin at mvath.de)

This project is under the BSD license 2.0 (“3-clause BSD license”).SPDX-License-Identifier: BSD-3-Clause

Init and management script for mounting rewritable squashfs-compressed data

This is the successor of thesquash_dir project

• https://github.com/vaeth/squash_dir/

It is actually a ground-up rewrite of that project in perl with ahighly improved control interface.

One of its aims is to be init system independent.For openrc and systemd ready-to-use init/service-files are provided;I gladly add init-files for other init systems if I receive (tested) patches.

squashmount works with systemd in “standard” setups, but due toconceptional bugs of systemd in some setups (see section “Installation”),systemd is no longer officially supported.

This issquashmount, a generic initscript, user interface and managementtool for keeping directories compressed by squashfs but simultaneously keepthem writable using some of (depending on the configuration and what isavailable):

• overlay (AKA overlayfs for linux kernels 3.18 and newer)
• overlayfs, a variant for older linux kernels, seehttp://git.kernel.org/?p=linux/kernel/git/mszeredi/vfs.git
• aufs, seehttp://aufs.sourceforge.net
• fuse-overlayfs, seehttps://github.com/containers/fuse-overlayfs
• unionfs-fuse, seehttp://podgorny.cz/moin/UnionFsFuse(unionfs-fuse-0.25 or newer is required)
• unionfs, seehttp://www.fsl.cs.sunysb.edu/project-unionfs.html
• funionfs, seehttp://bugs.gentoo.org/show_bug.cgi?id=151673

Sincesquashmount v15.0.0 there is also a choice between

• squashfs (the linux kernel module)
• squashfuse, the fuse module, seehttps://github.com/vasi/squashfuse

The idea is that, as a rule, on shutdown the data is recompressed(and the temporary modified data removed).This approach is originally due to synss' script fromhttp://forums.gentoo.org/viewtopic-t-465367-highlight-.htmlIn that forum thread you can also ask for help about this project.

For some mount points different rules than “resquash and delete on umount”might be desired (see the examples section below), and moreover,it might be necessary to override these rules temporarily.For such things a powerful user interface is provided.

This project can be useful for any linux distribution.Historically, the main motivations was to keep the Gentoo main repositorycompressed. This is still one of the most striking examples:

If the Gentoo main repository (/usr/portage or/var/db/repos/gentoo)is compressed withsquashmount (withoutDISTDIR which you should storesomewhere else when using this script), the required disk space is onlyabout 50-100 MB (depending on your compression method), instead of200-400 MB (or much higher, the actual space requirementdepending essentially on the filesystem, e.g. how inodes are used).Usually, also the access is much faster.

It is possible to combinesquashmount with portage'ssync-type = squashdelta to mount the Gentoo repository writable.

The script requires of course that squashfs support is activated in thekernel (and supports the COMPRESSION method), that the mksquashfs toolis available, and also that some of the above mentioned unionfs-type toolsis available and supported by the kernel.

Since v17.0.0/v10.0.0 ,squashmount defaults to the COMPRESSION methodzstd/lz4. This method is available only in linux-4.14/3.19 or higheror in squashfuse-0.1.101_alpha20170917/0.1.100_alpha20140523 or higher.So take care to either use a sufficiently new kernel/squashfuse versionor to change the default!

Moreover, you need a decently new version ofperl5 together with some ofits standard modules (which might need to be installed separately if yourperl5 version should be too old). Decently new perl versions should have theTERM::ANSIColor module; you need this if you want to see nicelycolored output.

It is also strongly recommended to install theFile::Which module(although there are some fallbacks if it is not available).

If you want that the hard status line is set, also the title script fromhttps://github.com/vaeth/runtitle (version >=2.3) is required in your path.

If you are a Gentoo user, you can just emergesquashmount from themv overlay.

Otherwise you just have to copybin/squashmount into/usr/bin/squashmountor any other directory of your$PATH.For zsh completion support also copyzsh/_squashmount into a directory ofyour zsh's$fpath.

It is strongly recommended to put

• alias squashmount='noglob squashmount'

into your~/.zshrc or/etc/zsh/zshrc or/etc/zshrc,so that things like

• squashmount start *

will work in your zsh as intended without the need to quote *.(I assume that you do not use any poor shell instead of zsh.) ;)

If you userevdep-rebuild from Gentoo or similar distributions, andif you use the default naming scheme, it is recommended to copy the content ofetc/revdep-rebuild into/etc/revdep-rebuild to cancel duplicate or obsoletepaths search of revdep-rebuild.

Foropenrc support copyopenrc/init.d/squashmount to/etc/init.d/squashmount and activate it in the usual way.For systemd-support copysystemd/system/squashmount.service to yoursystemd unit folder (pkg-config --variable=systemdsystemunitdir systemd,usually/lib/systemd/system or/usr/lib/systemd/system)and activate it in the usual way (or e.g. copy into/etc/systemd/system)

If you usesystemdbe sure to compile themount binarywithsystemd support(this should be the case in most distributions providing systemd; in Gentoothis means to enableUSE=systemd for the util-linux package. If you compileutil-linux manually, make sure to pass--with-systemd to./configure).In this case,systemd will probably work for you in “standard” setups.Withsystemd-219 (or newer?) and some unusual setups like--make-sharedon some partitions, it can happen nevertheless thatmount appears to work,but actually nothing is mounted ifsystemd is in use. This is related withthe fact thatsystemd tries to control all mounts instead of letting thekernel do it alone. Of course, this breaks tools likesquashmountcompletely.Bugsystemd upstream about such problems, but not me: I am not planningto add hacks to fix the breakage introduced by some ill-conceivedsystemdconcepts.

Forsystemd, you should set an appropriate timeout: There is no generalrule how long compression can take “maximally”, so the timeout is set toinfinity, by default. It is strongly recommended to set this to a realisticvalue for you system and setting by giving a (generous) upper estimate foryour needs by copying the fileetc/system/squashmount.service.d/timeout.confto/etc/systemd/squashmount/service.d/timeout.conf and editing appropriately.If you copied the main script not to/usr/bin/squashmount, you shouldput into the same directory a file with appropriate modified paths.For instance, if you copied the main script to/sbin/squashmount thencreate/etc/systemd/system/squashmount.service.d/exec.conf with the content

[Service]ExecStart=ExecStart=/sbin/squashmount startExecStop=ExecStop=/sbin/squashmount -f --lsof=0 stop

Also copytmpfiles.d/squashmount.conf to/usr/lib/tmpfiles.d, although thisis not absolutely necessary (squashmount will create the correspondingdirectories anyway).If you use an init-system which does not mount /run as a ramdisk,you should cleanup /run/squashmount on every fresh start beforecallingsquashmount start.Depending on your init-system, a way to achieve this might be to change thefirst letter in the crucial line in/usr/lib/tmpfiles.d/squashmount.conffromd toD! and to make sure that the processing of/usr/lib/tmpfiles.dtakes place before callingsquashmount start.(Since accidental cleaning can have very inconvenient consequences, andcurrently only systemd supports theD! syntax,d is the default.)See sectionEmergency Case what to do if/run/squashmountis removed accidentally anyway.If you usefind_cruft, you might want to copy the content oflib/find_cruft to/usr/lib/find_cruft or/etc and adapt it to your needs.

If you plan to use portage'ssync-type = squashdelta, you might want to copythe content ofetc/portage/repo.postsync.d to/etc/portage/repo.postsync.dNote that the hook-file in this directory treats the mount point “gentoo”specially! See the example configuration inetc/squashmount.pl how tosetup an appropriate mount point “gentoo” for this setting.

In all cases you have to copylib/squashmount.pl to/etc/squashmount.pland adapt it to your need! This is an essential point of squashmount,and it is impossible to use squashmount without setting up the configuration.You can optionally also copylib/squashmount.pl to/lib/squashmount.pl or/usr/lib/squashmount.pl to provide a system-wide example config.Alternatively, you can also modify that file to use it as a fallback if/etc/squashmount.pl is not readable.

Essentially, the init-system (or you) has to call

• squashmount start

on start and

• squashmount -f --lsof=0 stop

on shutdown (at a time when the local filesystems are already/still mounted).The provided installation files for systemd and openrc do just this.

This will cause all configured mount points to be mounted/umountedcorrespondingly. When umounting, by default the modified data isrecompressed into the squash-files (but this can be customized).

The configuration of the mount points happens in the file /etc/squashmount.plThis is a perl file, so you can use perl code in this file to source otherfiles at your discretion.

The provided example configuration file etc/squashmount.pl is ratherrealistic if you are a Gentoo user: It provides the following mount points

• (a) guest: A guest user's home directory /home/guest
• (b) tex: The installed files from texlive /usr/share/texmf-dist
• (c1) portage: The Gentoo repository /usr/portage
• (c2) gentoo: The Gentoo repository when using sync-type = squashdelta
• (d) db: The Gentoo database of installed packages /var/db

Further mount points are in the example config-file but not listed here.

For all the mount points it is reasonable to use squashmount with them fordifferent reasons:

• (a) The guest-user should be able to modify data in /home/guest, but itschanges should usually be forgotten. (Sometimes you will not want toforget these changes, e.g. when you want to update the “default”home directory which the user sees; see below how to do this).

• (b) The tex directory is huge, and it saves considerable space to keep itcompressed on disk.

• (c1) Keeping the Gentoo repository compressed does not only save an enormousamount of disk space but actually also speeds up portage considerable,because the disk access is faster on a single (squashed) file.Moreover, after changes with eix --sync you might want to compare thenew files with previous versions in the squashed file which you can stillaccess when you use squashmount.

• (c2) When you prefer to use portage'ssync-type = squashdelta, you alreadyhave the advantage of a compressed portage tree. However, you might want touse squashmount to make this portage tree writable (e.g. in order totemporarily fix a broken Manifest file locally).

• (d) The db directory is short but its data is very sensible and thenumber files is huge: Keeping it in a compressed file gives a lotof disk space and speedup, and it makes sense to keep a compressed backupof the last mounted version.

In these examples, additional features of squashmount are used inthe example configuration:

1. For (a), it is obviously necessary to use a different treatment:Normally, the squash-file should not be generated, and the temporary datashould be removed.A similar remark holds for (c2).

2. For (d), squashmount will keep a backup even of the squash-file forthe db directory.

3. squashing of (c1) and (d) should happen only when a certainthreshold of changes to the data is reached.The modified data will survive the reboot even if it is not resquashed,but it takes more diskspace of course, and there is no readonly version ofthe corresponding files.

4. No resquash of the tex directory when only certain files (like theautomatically generatedls-R file) were updated:In fact, if the only changes made to the directoy are in these files(it is optionally also checked that their content is not changed),the directory will be cleared when umounting/remounting/rebooting.

You can also call squashmount at runtime to resquash or clean certaindirectories manually or to change states for the above “default” actionson future umounts. For instance, if you changed the skeleton of theguest user in (a) you can call

• squashmount --no-kill remount guest

to force immediate regeneration of the squashed file from the directory.Moreover, you can call squashmount to temporarily change the state of mounteddirectories. For example, if you want to change temporarily that the guestuser's data is saved on remount, use

• squashmount --no-kill set guest

A changed state will remain active until reset, restart, or stop is executed.Another example: If a lot of data would be resquashed at the next umount,but you want to reboot urgently, just call

• squashmount --no-squash set

before rebooting.Conversely, if you want to squash the portage mount point despite itsthreshold is not reached (e.g. because you plan to make an experimentalchange which you plan to undo later), you can call

• squashmount --squash restart portage

If afterwards you made your experimental change and want to undo it, call

• squashmount --kill restart portage

If your changes take a longer time and you want to make sure that you do notforget to call the above command, you can temporarily change the state:

• squashmount --kill set portage

Normally, this setting will survive a remount. In order to reset to theoriginal setting after remounting, use option-r withremount:

• squashmount -r remount portage

This is the same as calling

• squashmount remount portagesquashmount reset portage

The above examples should perhaps be enough to give you an impression howto usesquashmount. To get an exact description of the user interface andof the config file format just execute:

• squashmount man

If you accidentally removed or corrupted/run/squashmount, e.g. due to a bugin squashmount itself or in its configuration or if the init-system wasmisconfigured and removed /run/squashmount after callingsquashmount start,you should try to umount unconditionally.You can instruct squashmount to do this with

• squashmount -fI umount

or (if you do not want resquashing in such a situation):

• squashmount -nfI umount

This will work reliably unless you used temporary directories in your setup.It will even work with temporary directories if their name is still storedin /run/squashmount, i.e. if the information there was not lost completely.

It is in general rather safe to squash a directory, even a rather vital one:Even if e.g. you boot from a kernel which has no support for some ofoverlay|overlayfs|aufs|unionfs-fuse|unionfs |funionfsto make the directory writable,squashmount will mount it at least asread-only (usingmount --bind if necessary).Moreover, if everything goes wrong you can still use theunsquashfs toolto unpack the directory manually.Probably the only danger in packing “strange” directories are special files,hard links (this information will usually get lost), or special deviceswhich are perhaps not supported by the used tools.

Since version 3.0, unless configured otherwise,squashmount will attemptto modprobe the modules

• squashfs
• aufs
• fuse
• overlay
• overlayfs
• unionfs

when they are required; optionally/alternatively, also the correspondingkernel option can be checked in/sys/module or/proc/config.gz;also the existence of required binaries can be checked before actuallythe mounting is attempted.It depends on your setting whether this is done and/or whether in caseof failure the corresponding tool is skipped tacitly without attemptingto mount.

If no tool mounts successfully, it is attempted to usemount --bind to getthe directory at least readonly on the expected place, so even inthis bad situation (which probably only happens if you boot from anexperimental kernel or a brand new kernel without corresponding support)you can still access the directory read-only. Hence, also rather vitaldirectories can be compressed as long as it is not vital to write to them(and as long as the relevant programs for mounting etc. are not containedwithin these directories, of course).

If you rely only on the tool squashfuse and either unionfs-fuse or funionfs(which are all based on the fuse userspace file system), thensquashmount can also be used with user permissions.To support selecting only these tools and appropriate files and dirs fromthe calling user, the option--user is supported since squashmount v15.0.0.For details, callsquashmount man and look for the option --user.

It is recommended to use a new version than 4.3 of squashfs-tools whichhas the-quiet option. Cf. the description of--mksquash-verboseand the$mksquash_verbose variable in./squashmount man for details.As a Gentoo user you can install such a recent version from the mv overlay.

However, this recommendation is only an eye-candy:squashmount will workwithout any problems also with an unpatched version ofsquashfs-tools;just the display when squashing will not be so nice.