In computing,process substitution is a form ofinter-process communication that allows the input or output of a command to appear as a file. The command is substituted in-line, where a file name would normally occur, by thecommand shell. This allows programs that normally only accept files to directly read from or write to another program.
Process substitution was available as a compile-time option forksh88, the 1988 version of theKornShell fromBell Labs.[1] Therc shell provides the feature as "pipeline branching" inVersion 10 Unix, released in 1990.[2] TheBash shell provided process substitution no later than version 1.14, released in 1994.[3]
The following examples use KornShell syntax.
TheUnixdiff command normally accepts the names of two files to compare, or one file name and standard input. Process substitution allows one to compare the output of two programs directly:
$diff<(sortfile1)<(sortfile2)
The<(command) expression tells the command interpreter to runcommand and make its output appear as a file. Thecommand can be any arbitrarily complex shell command.
Without process substitution, the alternatives are:
$sortfile2>/tmp/file2.sorted$sortfile1|diff-/tmp/file2.sorted$rm/tmp/file2.sorted
$mkfifo/tmp/sort2.fifo$sortfile2>/tmp/sort2.fifo&$sortfile1|diff-/tmp/sort2.fifo$rm/tmp/sort2.fifo
Both alternatives are more cumbersome.
Process substitution can also be used to capture output that would normally go to a file, and redirect it to the input of a process. The Bash syntax for writing to a process is>(command). Here is an example using thetee,wc andgzip commands that counts the lines in a file withwc -l and compresses it withgzip in one pass:
$tee>(wc-l>&2)<bigfile|gzip>bigfile.gz
The main advantages of process substitution over its alternatives are:
Under the hood, process substitution has two implementations. On systems which support/dev/fd (most Unix-like systems) it works by calling thepipe() system call, which returns a file descriptor$fd for a new anonymous pipe, then creating the string/dev/fd/$fd, and substitutes that on the command line. On systems without/dev/fd support, it callsmkfifo with a new temporary filename to create a named pipe, and substitutes this filename on the command line. To illustrate the steps involved, consider the following simple command substitution on a system with/dev/fd support:
$difffile1<(sortfile2)
The steps the shell performs are:
/dev/fd/63; you can see it with a command likeecho <(true).sort file2 in this case), piping its output to the anonymous pipe.diff file1 /dev/fd/63.For named pipes, the execution differs solely in the creation and deletion of the pipe; they are created withmkfifo (which is given a new temporary file name) and removed withunlink. All other aspects remain the same.
The "files" created are notseekable, which means the process reading or writing to the file cannot performrandom access; it must read or write once from start to finish. Programs that explicitly check the type of a file before opening it may refuse to work with process substitution, because the "file" resulting from process substitution is not aregular file.Additionally, up to Bash 4.4 (released September 2016), it was not possible to obtain the exit code of a process substitution command from the shell that created the process substitution.[4]