Repository files navigation

Source:https://github.com/mbland/go-script-bash

A./go script aims to abstract away many of the steps needed to develop (andsometimes deploy) a software project. It is a replacement for READMEs and otherdocuments that may become out-of-date, and when maintained properly, shouldprovide a cohesive and discoverable interface for common project tasks.

For a five-minute overview of the framework, seeMike Bland's go-script-bashlightning talk at Surge 2016.

• Introduction
• Environment setup
• How to use this framework
• Feedback and contributions
• Installing Bash
• Open Source
• Prior work

The./go script idea came from Pete Hodgson's blog postsIn Praise of the./go Script: Part I andPart II. To paraphrase Pete'soriginal idea, rather than dump project setup, development, testing, andinstallation/deployment commands into aREADME that tends to get stale, orrely on oral tradition to transmit project maintenance knowledge, automate thesetasks by encapsulating them all inside a single script in the root directory ofyour project source tree, conventionally named "go". Then the interface tothese tasks becomes something like./go setup,./go test, and./go deploy.Not only would this script save time for people already familiar with theproject, but it smooths the learning curve, prevents common mistakes, and lowersfriction for new contributors. This is as desirable a state for Open Sourceprojects as it is for internal ones.

No. The./go script convention in general and this framework in particular arecompletely unrelated to theGo programming language. In fact, theactual./go script can be named anything. However, thego command from theGo language distribution encapsulates many common project functionsin a similar fashion.

Of course, the danger is that this./go script may become as unwieldy as theREADME it's intended to replace, depending on the project's complexity. Evenif it's heavily used and kept up-to-date, maintenance may become an intensive,frightening chore, especially if not covered by automated tests. Knowing whatthe script does, why it does it, and how to run it may become more and morechallenging—resulting in the same friction, confusion, and fear the script wastrying to avoid.

The./go script framework makes it easy to provide a uniform and easy-to-useproject maintenance interface that fits your project perfectly regardless of themix of tools and languages, then it gets out of the way as fast as possible. Thehope is that bymaking the right thing the easy thing,scripts using the framework will evolve and stay healthy along with the rest ofyour project sources, which makes everyone working with the code less frustratedand more productive all-around.

This framework accomplishes this by:

• encouraging modular, composable./go commands implemented as individualscripts—in the language of your choice!
• providing a set of builtin utility commands and shell command aliases—see./go help builtins and./go help aliases
• supporting automatic tab-completion of commands and arguments through alightweight API—see./go help env and./go help complete
• implementing a quick, flexible, robust, and convenient documentationsystem—document your script in the header, and help shows up automatically as./go help my-command! See./go help help.

Plus, its own tests serve as a model for testing command scripts of all shapesand sizes.

The inspiration for this model (and initial implementation hints) came fromSamStephenson'srbenv Ruby version manager.

It's the ultimate backstage pass! It's the default shell for mostmainstream UNIX-based operating systems, easily installed on other UNIX-basedoperating systems, and is readily available even on Windows.

Yes. It is an explicit goal to make it as easy to use the framework on Windowsas possible. SinceGit for Windows in particular ships with Bash aspart of its environment, and Bash is available within Windows 10 as part of theWindows Subsystem for Linux (Ubuntu on Windows), it's more likely thannot that Bash is already available on a Windows developer's system. It's alsoavailable from theMSYS2 andCygwin environments.

Of course there are many common tools that may be used for managing projecttasks. For example:Make,Rake,npm,Gulp,Grunt,Bazel,and the Go programming language'sgo tool. There are certainly more powerfulscripting languages:Perl,Python,Ruby, and evenNode.jsis a possibility. There are even more powerful shells, such as theZ-Shell and thefish shell.

The./go script framework isn't intended to replace all those other tools andlanguages, but to make it easier to use each of them for what they're good for.It makes it easier to write good, testable, maintainable, and extensible shellscripts so you don't have to push any of those other tools beyond their naturallimits.

Bash scripting isreally good for automating a lot of traditional command linetasks, and it can be pretty awkward to achieve the same effect using othertools—especially if your project uses a mix of languages, where using a toolcommon to one language environment to automate tasks in another can get weird.(Which is part of the reason why there are so many build tools tailored todifferent languages in the first place, to say nothing of the differentlanguages themselves.)

If you want to incorporate different scripting languages or shells into yourproject maintenance, this framework makes it easy to do so. However, by startingwith Bash, you can implement a./go init command to check that these otherlanguages or shells are installed and either install them automatically orprompt the user on how to do so. Since Bash is (almost certainly) alreadypresent, users can run your./go script right away and get the setup or hintsthat they need, rather than wading through system requirements and documentationbefore being able to do anything.

Even if./go init tells the user "go to this website and install this otherthing", that's still an immediate, tactile experience that triggers a rewardresponse and invites further exploration. (Think ofZork and the first"open mailbox" command.)

The real question is: Wherecan't you run it?

The core framework is written 100% inBash and it's beentested under Bash 3.2, 4.2, 4.3, and 4.4 across OS X, Ubuntu Linux, Arch Linux,Alpine Linux, FreeBSD 9.3, FreeBSD 10.3, and Windows 10 (using all theenvironments described in the "Will this work on Windows?" section above).

Actually, yes. See theStandalone mode section below.

Also see the following question...

Yes. You can share one copy of the go-bash-framework sources, and even havecommon code in thelib/ directory, but set each script to use its own commandscripts dir.

This may be especially useful if you're writing astandaloneprogram, in which one script provides the actual program interface, and theother provides the development-only interface.

The project's own./go test command does it all. Combined with automatictab-completion enabled by./go env and pattern-matching via./go glob, the./go test command provides a convenient means of selecting subsets of testcases while focusing on a particular piece of behavior. (See./go help test.)

The tests are written usingmbland/bats, an optimized version of SamStephenson's Bash Automated Testing System (BATS). Code coveragecomes fromSimon Kagstrom'skcov code coverage tool, which not onlyprovides code coverage for Bash scripts (!!!) but can push the results toCoveralls!

To run a./go script that uses this module, or to add it to your own project,you must haveBash version 3.2 or greater installed on yoursystem. Runbash --version to make sure Bash is in yourPATH and is acompatible version. You should see output like this:

GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15)Copyright (C) 2007 Free Software Foundation, Inc.

If you do not see this, follow the instructions in theInstallingBash section later in this document.

Note: While Bash is required to run this framework, your individual commandscripts can be in any other interpreted language installed on the host system.

First you'll need a copy of this framework available in your project sources.The most expedient way to bootstrap your program is to use thego-templatefile as a starting point (replacingcurl withwget,fetch, or whichever tool you prefer):

$ curl https://raw.githubusercontent.com/mbland/go-script-bash/master/go-template>./go$ chmod ugo+rx ./go

You may rename this file whatever you wish (i.e. it doesn't have to be named./go), update its documentation and variables to fit your project, and checkit into your project repository. See thego-template comments for details.

If you'd prefer to download a copy of the framework and check it into yoursources, versioned archives are available from thego-script-bash Releasespage. The archives for the current release are:

• https://github.com/mbland/go-script-bash/archive/v1.7.0.tar.gz
• https://github.com/mbland/go-script-bash/archive/v1.7.0.zip

You can also add this repository to your project as a Git submodule:

$ git submodule add https://github.com/mbland/go-script-bash<target-dir>$ git commit -m'Add go-script-bash framework'$ git submodule update --init

where<target-dir> is any point inside your project directory structure thatyou prefer.

If you're not usinggo-template, create a bash script in the root directory ofyour project to act as the main./go script. This script need not be namedgo, but it must contain the following lines, with@go "$@" as the last lineof the script:

."${0%/*}/go-core.bash""scripts"@go"$@"

where:

• ${0%/*} produces the path to the project's root directory based on the pathto the./go script
• ${0%/*}/go-core.bash produces the path to the framework'sgo-core.bashfile within your project's copy of the framework (adjusted to reflect whereyour copy ofgo-script-bash actually resides)
• scripts is the path to the directory holding your project's command scriptsrelative to the project root (it can be any name you like)

The./go script changes to the project root directory before executing anycommands. That means every command script you write will also run within theproject root directory, so every relative file and directory path will beinterpreted as relative to the project root.

Your project structure may look something like this:

project-root/  go - main ./go script  lib/ - publicly-exported modules (if the project is a go-bash-script plugin)  scripts/ (or bin/) - project (or plugin) ./go command scripts    lib/ - project-specific Bash library modules (see "Modules" section)    plugins/ - (optional) third-party command scripts (see `./go help plugins`)      .../        bin/ - plugin ./go command scripts        lib/ - publicly-exported Bash library modules (see "Modules" section)    go-script-bash/      go-core.bash - top-level functions      lib/ - publicly-exported Bash library modules (see "Modules" section)      libexec/ - builtin ./go command scripts

This structure implies that the first line of your./go script will be:

."${0%/*}/scripts/go-script-bash/go-core.bash""scripts"

The following variables are set by the framework based on the above example(note there are many other variables set ingo-core.bash and elsewhere; see./go help vars):

• _GO_ROOTDIR:/absolute/path/to/project-root
• _GO_CORE_DIR:/absolute/path/to/project-root/scripts/go-script-bash
• _GO_SCRIPTS_DIR:$_GO_ROOTDIR/scripts
• _GO_PLUGINS_DIR:/absolute/path/to/project-root/plugins

For plugins,_GO_ROOTDIR and_GO_SCRIPTS_DIR will be scoped to the rootdirectory of the plugin installation; the other variables will remain the same.See./go help plugins for more details.

Each command script for your project residing in thescripts directory mustadhere to the following conditions:

• No filename extensions.
• It must be executable, with a#! (a.k.a. "she-bang") line. The interpretername will be parsed from this line, whether it is an absolute path(#!/bin/bash) or is of the form:#!/usr/bin/env bash.
• Ifscripts/parent is a command script, subcommand scripts must reside withina directory named:scripts/parent.d.

Scripts can use any interpreted language available on the host system; theyneed not be written in Bash. Bash scripts will be sourced (i.e. imported intothe same process running the./go script itself). Other languages will use thePATH environment variable to discover the interpreter for the script.

See./go help commands for details on the algorithm used to discover commandscripts for execution.

The builtin./go help command will parse command script summaries and helptext from the header comment block of each script. Run./go help help to learnmore about the formatting rules.

By evaluating the value of./go env - within your shell, all builtin commandsand aliases provide automatic tab completion of file, directory, and otherarguments. If an implementation isn't available for your shell (withinlib/internal/env/), it's very easy to add one. Feel free to open an issue or,better yet,send a pull request!

To learn the API for adding tab completion to your own command scripts, run./go help complete. You can also learn by reading the scripts for the builtincommands.

If you wish to use the framework to write a standalone program, rather than aproject-specific development script, set_GO_STANDALONE in your top-levelscript to prevent alias commands, builtin commands, and plugin commands fromshowing up inhelp output or from being offered as tab completions. (helpwill still appear as a top-level tab completion.) All of these commands willstill be available, but users won't be presented with them directly.

_GO_STANDALONE also prevents the script from settingPWD to_GO_ROOTDIR,enabling the script to process relative file path arguments anywhere in the filesystem. Note that then you'll have to add_GO_ROOTDIR manually to any_GO_ROOTDIR-relative paths in your own scripts.

There are a number of possible methods available for sharing code betweencommand scripts. Some possibilities are:

• The generally preferred method is to use. $_GO_USE_MODULES to sourceoptional library modules; see theModules section.
• Include common code and constants in the top-level./go script, aftersourcinggo-core.bash and before calling@go.
• Source a file in the same directory that isn't executable.
• Source a file in a child directory that may not have a name of the form:parent.d.
• Source files from a dedicated directory relative to$_GO_ROOTDIR, e.g.:

  ."path/to/lib/common.sh"

• Subcommand scripts can source the parent command via:

  ."${BASH_SOURCE[0]%.d/*}"

Any script in any language can invoke other command scripts by running./go <command> [args..]. In Bash, however, you can also invoke the@gofunction directly as@go <command> [args...].

The@go,@go.printf, and@go.print_stack_trace functions are available tocommand scripts written in Bash, as Bash command scripts are sourced rather thanrun using another language interpreter.

A number of global variables defined and documented ingo-core.bash, allstarting with the prefix_GO_, are exported as environment variables andavailable to scripts in all languages (along with the globalCOLUMNSenvironment variable). Run./go vars to see them all along with their values,and run./go help vars for more details.

You can add third-party plugin command scripts to theplugins subdirectory ofyour scripts directory. Run./go help plugins for more information.

You can import optional Bash library code from the core framework, third-partyplugins, or your own project's scripts directory by sourcing the_GO_USE_MODULES script. For example, to import the core logging utilities:

."$_GO_USE_MODULES"'log'

Run./go help modules and./go modules --help for more information.

The core librarylog module provides functions for standard loggingfacilities. For example:

@go.log INFO Hello, World!@go.log ERROR Goodbye, World!

For more information, run./go modules --help log.

The assertions and helpers from the test suite have been extracted into thelib/bats libraries. While these are not modules you can import with_GO_USE_MODULES, they are completely independent of the rest of the coreframework and you may source them in your own Bats tests. (Whether or not thesewill ever become a separate library remains an open question.)

Variables, helper functions, and assertions for testing features based on thecore framework are available in thelib/testing directory. Thelib/bats-mainlibrary makes it easy to write a./go test command script with the sameinterface and features as the core framework's./go test command.

Read the comments from each file for more information.

Thekcov-ubuntu module provides therun_kcov function that will download andcompilekcov, then runkcov with the original./go command linearguments to collect test coverage. Only available on Ubuntu Linux for now,hence the name. Run./go modules --help kcov-ubuntu for more information andseescripts/test for an example of how it may be used.

Feel free tocomment on or file a new GitHub issue or otherwise ping@mbland with any questions or comments you may have, especially if thecurrent documentation hasn't addressed your needs.

If you'd care to contribute to this project, be it code fixes, documentationupdates, or new features, please read theCONTRIBUTING file.

If you're using a flavor of UNIX (e.g. Linux, OS X), you likely already have asuitable version of Bash already installed and available. If not, use yoursystem's package manager to install it.

On Windows, theGit for Windows,MSYS2 andCygwindistributions all ship with a version of Bash. On Windows 10, you can also usetheWindows Subsystem for Linux.

Once you've installedbash, yourPATH environment variable must includeits installation directory. On UNIX, you can add it in the appropriateinitialization file for your shell; look up your shell documentation for details.

On Windows, in most cases, you'll use the terminal program that ships with Gitfor Windows, MSYS2, or Cygwin, or you'll invoke the Windows System for Linuxenvironment by enteringbash in a built-in Command Prompt window. Theseterminals automatically setPATH so that Bash is available.

However, if you want to use the Git, MSYS2, or Cygwinbash from the built-inCommand Prompt window, open theStart menu and navigate toWindowsSystem > Control Panel > System and Security > System > Advanced systemsettings. Click theEnvironment Variables... button, selectPATH, andadd the directory containing yourbash installation. The likely paths for eachenvironment are:

• Git:C:\Program Files\Git\usr\bin\
• MSYS2:C:\msys64\usr\bin\
• Cygwin:C:\cygwin64\bin\

To use one of these paths temporarily within a Command Prompt window, you canrun the following:

C:\path\to\my\go-script-bash> set PATH=C:\Program Files\Git\usr\bin\;%PATH%# To verify:C:\path\to\my\go-script-bash> echo %PATH%C:\path\to\my\go-script-bash> where bash# To run the tests:C:\path\to\my\go-script-bash> bash ./go test

It should not be necessary to set Bash as your default shell. On Windows,however, you may wish to execute thebash command to run it as your shellbefore executing the./go script or any other Bash scripts, to avoid having torun it asbash ./go every time.

Most of the framework as-is does not require any other external tools. However,in order for the automatic command help and output formatting to work, you'llneed the following utilities installed:

• tput (ncurses) on Linux, OS X, UNIX
• mode.com should be present on Windows

To use theget file builtin, eithercurl,wget, orfetch must beinstalled on your system.get git-repo requiresgit, naturally.

This software is made available asOpen Source software under theISC License. For the text of the license, see theLICENSEfile.

This is a Bash-based alternative to the18F/go_script Rubyimplementation.