- Notifications
You must be signed in to change notification settings - Fork129
Non-blocking external commands in Go with streaming output
License
go-cmd/cmd
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
This package is a small but very useful wrapper aroundos/exec.Cmd that makes it safe and simple to run external commands in highly concurrent, asynchronous, real-time applications. It works on Linux, macOS, and Windows. Here's the basic usage:
import ("fmt""time""github.com/go-cmd/cmd")funcmain() {// Start a long-running process, capture stdout and stderrfindCmd:=cmd.NewCmd("find","/","--name","needle")statusChan:=findCmd.Start()// non-blockingticker:=time.NewTicker(2*time.Second)// Print last line of stdout every 2sgofunc() {forrangeticker.C {status:=findCmd.Status()n:=len(status.Stdout)fmt.Println(status.Stdout[n-1])}}()// Stop command after 1 hourgofunc() {<-time.After(1*time.Hour)findCmd.Stop()}()// Check if command is doneselect {casefinalStatus:=<-statusChan:// donedefault:// no, still running}// Block waiting for command to exit, be stopped, or be killedfinalStatus:=<-statusChan}
That's it, only three methods:Start
,Stop
, andStatus
. When possible, it's better to usego-cmd/Cmd
thanos/exec.Cmd
becausego-cmd/Cmd
provides:
- Channel-based fire and forget
- Real-time stdout and stderr
- Real-time status
- Complete and consolidated return
- Proper process termination
- 100% test coverage, no race conditions
As the example above shows, starting a command immediately returns a channel to which the final status is sent when the command exits for any reason. So by default commands run asynchronously, but running synchronously is possible and easy, too:
// Run foo and block waiting for it to exitc:=cmd.NewCmd("foo")s:=<-c.Start()
To achieve similar withos/exec.Cmd
requires everything this package already does.
It's common to want to read stdout or stderrwhile the command is running. The common approach is to callStdoutPipe and read from the providedio.ReadCloser
. This works but it's wrong because it causes a race condition (thatgo test -race
detects) and the docs say it's wrong:
It is thus incorrect to call Wait before all reads from the pipe have completed. For the same reason, it is incorrect to call Run when using StdoutPipe.
The proper solution is to set theio.Writer
ofStdout
. To be thread-safe and non-racey, this requires further work to write while possibly N-many goroutines read.go-cmd/Cmd
has done this work.
Similar to real-time stdout and stderr, it's nice to see, for example, elapsed runtime. This package allows that:Status
can be called any time by any goroutine, and it returns this struct:
typeStatusstruct {CmdstringPIDintCompleteboolExitintErrorerrorRuntimefloat64// secondsStdout []stringStderr []string}
Speaking of that struct above, Go built-inCmd
does not put all the return information in one place, which is fine because Go is awesome! But to save some time,go-cmd/Cmd
uses theStatus
struct above to convey all information about the command. Even when the command finishes, callingStatus
returns the final status, the same final status sent to the status channel returned by the call toStart
.
os/exec/Cmd.Wait can block even after the command is killed. That can be surprising and cause problems. Butgo-cmd/Cmd.Stop
reliably terminates the command, no surprises. The issue has to do with process group IDs. It's common to kill the command PID, but usually one needs to kill its process group ID instead.go-cmd/Cmd.Stop
implements the necessary low-level magic to make this happen.
In addition to 100% test coverage and no race conditions, this package is actively used in production environments.
Brian Ip wrote the original code to get the exit status. Strangely, Go doesn't just provide this, it requires magic likeexiterr.Sys().(syscall.WaitStatus)
and more.
MIT © go-Cmd.
About
Non-blocking external commands in Go with streaming output