Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork75
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
License
shellspec/shellspec
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
ShellSpec is afull-featuredBDD unit testing framework for dash, bash, ksh, zsh andall POSIX shells that provides first-class features such as code coverage, mocking, parameterized test, parallel execution and more. It was developed as a dev/test tool forcross-platform shell scripts and shell script libraries. ShellSpec is a new modern testing framework released in 2019, but it's already stable enough. With lots of practical CLI features and simple yet powerful syntax, it provides you with a fun shell script test environment.
Version 0.28.0 has a lot of enhancements in the CLI. It is basically compatible, but there are some changes that you need to be aware of. SeeMigration Guide to Version 0.28.0 for details.
Thank you for your interest in ShellSpec. Please visit 🚩the official website to know the impressive features!
Let's have fun testing your shell scripts! (TryOnline Demo on your browser).
Latest Update.
SeeCHANGELOG.md
NOTE: This documentation contains unreleased features. Check them in the changelog.
- Supported shells and platforms
- Requirements
- Installation
- Tutorial
- ShellSpec CLI
- Project directory
- Typical directory structure
- Options file
.shellspec
- project options file.shellspec-local
- user custom options file.shellspec-basedir
- specfile execution base directory.shellspec-quick.log
- quick execution logreport/
- report file directorycoverage/
- coverage reports directoryspec/
- (default) specfiles directory- <HELPERDIR> (default:
spec/
)
- Specfile (test file)
- DSL syntax
- Basic structure
- Pending, skip and focus
- Hooks
- Helpers
- Directives
- Mocking
- Support commands
- Tagging
- About testing external commands
- How to test a single file shell script
- spec_helper
- Self-executable specfile
- Use with Docker
- Extension
- Code Coverage
- For developers
bash
>=2.03,bosh/pbosh
>=2018/10/07,posh
>=0.3.14,yash
>=2.29,zsh
>=3.1.9dash
>=0.5.4,busybox ash
>=1.20.0,busybox-w32
,GWSH
>=20190627ksh88
,ksh93
>=93s,ksh2020
,mksh/lksh
>=R28,pdksh
>=5.2.14FreeBSD sh
,NetBSD sh
,NetBSD ksh
,OpenBSD ksh
,loksh
,oksh
Platform | Test |
---|---|
Linux (Debian, Ubuntu, Fedora, CentOS, Alpine, Busybox, OpenWrt) | GitHub Actions orDocker |
macOS (Default installed shells, Homebrew) | GitHub Actions |
Windows (Git bash, msys2, cygwin, busybox-w32, WSL) | GitHub Actions |
BSD (FreeBSD, OpenBSD, NetBSD) | Cirrus CI (FreeBSD) or Manual (Others) |
Unix (Solaris, AIX) | Manual only |
ShellSpec uses shell built-in commands and only few basicPOSIX-compliant commands tosupport wide range of environments (exceptkcov
for optional code coverage).
Currently used external (not shell builtins) commands:
cat
,date
,env
,ls
,mkdir
,od
(or not POSIXhexdump
),rm
,sleep
,sort
,time
ps
(use to auto-detect shells in environments that don't implement procfs)ln
,mv
(use only when generating coverage report)kill
,printf
(most shells except some are built-in)
curl -fsSL https://git.io/shellspec| sh
or
wget -O- https://git.io/shellspec| sh
NOTE:https://git.io/shellspec
is redirected toinstall.sh
The installation using the web installer is mainly intended for development use.For CI, it is recommended to use a specific version (tag) in git or archives to avoid unexpected failures.
Advanced installation / upgrade
curl -fsSL https://git.io/shellspec| sh -s -- --yes
curl -fsSL https://git.io/shellspec| sh -s 0.19.1
curl -fsSL https://git.io/shellspec| sh -s -- --switch
curl -fsSL https://git.io/shellspec| sh -s 0.18.0 --switch
Uninstall
- Delete the ShellSpec executable file [default:
$HOME/.local/bin/shellspec
]. - Delete the ShellSpec installation directory [default:
$HOME/.local/lib/shellspec
].
Other usage
$curl -fsSL https://git.io/shellspec| sh -s -- --helpUsage: [sudo] ./install.sh [VERSION] [OPTIONS...] or : wget -O- https://git.io/shellspec | [sudo] sh or : wget -O- https://git.io/shellspec | [sudo] sh -s -- [OPTIONS...] or : wget -O- https://git.io/shellspec | [sudo] sh -s VERSION [OPTIONS...] or : curl -fsSL https://git.io/shellspec | [sudo] sh or : curl -fsSL https://git.io/shellspec | [sudo] sh -s -- [OPTIONS...] or : curl -fsSL https://git.io/shellspec | [sudo] sh -s VERSION [OPTIONS...]VERSION: Specify install version and method e.g 1.0.0 Install 1.0.0 from git master Install master from git 1.0.0.tar.gz Install 1.0.0 from tar.gz archive . Install from local directoryOPTIONS: -p, --prefix PREFIX Specify prefix [default: $HOME/.local] -b, --bin BIN Specify bin directory [default: <PREFIX>/bin] -d, --dir DIR Specify installation directory [default: <PREFIX>/lib/shellspec] -s, --switch Switch version (requires installation via git) -l, --list List available versions (tags) --pre Include pre-release --fetch FETCH Force command to use when installing from archive (curl or wget) -y, --yes Automatic yes to prompts -h, --help You're looking at it
Arch Linux
Installation on Arch Linux from the AURShellSpec package usingaura
:
#Install the latest stable version$aura -A shellspec
Homebrew / Linuxbrew
#Install the latest stable version$brew tap shellspec/shellspec$brew install shellspec
basher / bpkg
Installation withbasher
The officially supported version is ShellSpec 0.19.1 and later.
#Install from master branch$basher install shellspec/shellspec#To specify a version (example: 0.19.1)$basher install shellspec/shellspec@0.19.1
Installation withbpkg
The officially supported version is ShellSpec 0.19.1 and later.
#Install from master branch$bpkg install shellspec/shellspec#To specify a version (example: 0.19.1)$bpkg install shellspec/shellspec@0.19.1
git / archive (source code)
Download from git or archive and create a symbolic link.
From git
$cd /SOME/WHERE/TO/INSTALL$git clone https://github.com/shellspec/shellspec.git$ln -s /SOME/WHERE/TO/INSTALL/shellspec/shellspec /EXECUTABLE/PATH/
From archive
$cd /SOME/WHERE/TO/INSTALL$wget https://github.com/shellspec/shellspec/archive/{VERSION}.tar.gz$tar xzvf shellspec-{VERSION}.tar.gz$ln -s /SOME/WHERE/TO/INSTALL/shellspec-{VERSION}/shellspec /EXECUTABLE/PATH/
Executable path: e.g./usr/local/bin/
,$HOME/bin/
Use make instead of symbolic link creation
Download from git or archive and usemake
command.
How to install.
Install to/usr/local/bin
and/usr/local/lib
sudo make install
Install to$HOME/bin
and$HOME/lib
make install PREFIX=$HOME
How to uninstall.
sudo make uninstall
make uninstall PREFIX=$HOME
For environments that do not support symbolic links
Download from git or archive and create the followingshellspec
file instead of the symbolic link.
$cat<<'HERE'>/EXECUTABLE/PATH/shellspec#!/bin/shexec /SOME/WHERE/TO/INSTALL/shellspec/shellspec "$@"HERE$chmod +x /EXECUTABLE/PATH/shellspec
SeeReleases page if you want to download distribution archive.
Just create your project directory and runshellspec --init
to setup your project
#Create your project directory,for example"hello".$mkdir hello$cd hello#Initialize$shellspec --init create .shellspec create spec/spec_helper.sh#Write your first specfile (of course you can use your favorite editor)$cat<<'HERE'>spec/hello_spec.shDescribe 'hello.sh' Include lib/hello.sh It 'says hello' When call hello ShellSpec The output should equal 'Hello ShellSpec!' EndEndHERE#Create lib/hello.sh$mkdir lib$touch lib/hello.sh#It will fail because the hellofunctionis not implemented.$shellspec#Write hello function$cat<<'HERE'>lib/hello.shhello() { echo "Hello ${1}!"}HERE#It will success!$shellspec
ShellSpec CLI runs specfiles with the shell runningshellspec
.Usually it is/bin/sh
that is the shebang ofshellspec
. If you runbash shellspec
, it will be bash.Include
files from specfile will be executed in the same shell as well.
The purpose of this specification is to allow ShellSpec to easily change multiple types of shellsand enable the development of cross-platform shell scripts that support multiple shells and environments.
If you want to test with a specific shell, use the-s
(--shell
) option.You can specify the default shell in the.shellspec
file.
NOTE: If you execute ashell script file (not a shell function) from within the specfile,its shebang will be respected. Because in that case, it will be run as an external command.The-s
(--shell
) option also has no effect.If you are testing a external shell script file, you can useWhen run script
orWhen run source
.These ignore the shebang of external shell script file and run in the same shell that runs specfile.
NOTE: Since version 0.28.0,getoptions is used to parse options,so all POSIX and GNU compatible option syntax can be used. For example, you can abbreviate a long option.
See more info:ShellSpec CLI
$shellspec -hUsage: shellspec [ -c ] [-C <directory>] [options...] [files or directories...] Using + instead of - for short options causes reverses the meaning -s, --shell SHELL Specify a path of shell [default: "auto" (the shell running shellspec)] --require MODULE Require a MODULE (shell script file) -O, --options PATH Specify the path to an additional options file -I, --load-path PATH Specify PATH to add to $SHELLSPEC_LOAD_PATH (may be used more than once) --helperdir DIRECTORY The directory to load helper files (spec_helper.sh, etc) [default: "spec"] --path PATH Set PATH environment variable at startup --{no-}sandbox Force the use of the mock instead of the actual command --sandbox-path PATH Make PATH the sandbox path instead of empty [default: empty] --execdir @LOCATION[/DIR] Specify the execution directory of each specfile | [default: @project] -e, --env NAME[=VALUE] Set environment variable --env-from ENV-SCRIPT Set environment variable from shell script file -w, --{no-}warning-as-failure Treat warning as failure [default: enabled] --{no-}fail-fast[=COUNT] Abort the run after first (or COUNT) of failures [default: disabled] --{no-}fail-no-examples Fail if no examples found [default: disabled] --{no-}fail-low-coverage Fail on low coverage [default: disabled] --failure-exit-code CODE Override the exit code used when there are failing specs [default: 101] --error-exit-code CODE Override the exit code used when there are fatal errors [default: 102] -p, --{no-}profile Enable profiling and list the slowest examples [default: disabled] --profile-limit N List the top N slowest examples [default: 10] --{no-}boost Increase the CPU frequency to boost up testing speed [default: disabled] --log-file LOGFILE Log file for %logger directive and trace [default: "/dev/tty"] --tmpdir TMPDIR Specify temporary directory [default: $TMPDIR, $TMP or "/tmp"] --keep-tmpdir Do not cleanup temporary directory [default: disabled] The following options must be specified before other options and cannot be specified in the options file -c, --chdir Change the current directory to the first path of arguments at the start -C, --directory DIRECTORY Change the current directory at the start **** Execution **** -q, --{no-}quick Run not-passed examples if it exists, otherwise run all [default: disabled] -r, --repair, --only-failures Run failure examples only (Depends on quick mode) -n, --next-failure Run failure examples and abort on first failure (Depends on quick mode) -j, --jobs JOBS Number of parallel jobs to run [default: 0 (disabled)] --random TYPE[:SEED] Run examples by the specified random type | <[none]> [specfiles] [examples] -x, --xtrace Run examples with trace output of evaluation enabled [default: disabled] -X, --xtrace-only Run examples with trace output only enabled [default: disabled] --dry-run Print the formatter output without running any examples [default: disabled] **** Output **** --{no-}banner Show banner if exist "<HELPERDIR>/banner[.md]" [default: enabled] --reportdir DIRECTORY Output directory of the report [default: "report"] -f, --format FORMATTER Choose a formatter for display | <[p]> [d] [t] [j] [f] [null] [debug] -o, --output FORMATTER Choose a generator(s) to generate a report file(s) [default: none] --{no-}color Enable or disable color [default: enabled if the output is a TTY] --skip-message VERBOSITY Mute skip message | <[verbose]> [moderate] [quiet] --pending-message VERBOSITY Mute pending message | <[verbose]> [quiet] --quiet Equivalent of --skip-message quiet --pending-message quiet --(show|hide)-deprecations Show or hide deprecations details [default: show] **** Ranges / Filters / Focus **** You can run selected examples by specified the line numbers or ids shellspec path/to/a_spec.sh:10 # Run the groups or examples that includes lines 10 shellspec path/to/a_spec.sh:@1-5 # Run the 5th groups/examples defined in the 1st group shellspec a_spec.sh:10:@1:20:@2 # You can mixing multiple line numbers and ids with join by ":" -F, --focus Run focused groups / examples only -P, --pattern PATTERN Load files matching pattern [default: "*_spec.sh"] -E, --example PATTERN Run examples whose names include PATTERN -T, --tag TAG[:VALUE] Run examples with the specified TAG --default-path PATH Set the default path where looks for examples [default: "spec"] You can specify the path recursively by prefixing it with the pattern "*/" or "**/" (This is not glob patterns and requires quotes. It is also available with --default-path) shellspec "*/spec" # The pattern "*/" matches 1 directory shellspec "**/spec" # The pattern "**/" matches 0 and more directories shellspec "*/*/**/test_spec.sh" # These patterns can be specified multiple times -L, --dereference Dereference all symlinks in in the above pattern [default: disabled] **** Coverage **** --covdir DIRECTORY Output directory of the Coverage Report [default: coverage] --{no-}kcov Enable coverage using kcov [default: disabled] --kcov-path PATH Specify kcov path [default: kcov] --kcov-options OPTIONS Additional Kcov options (coverage limits, coveralls id, etc) **** Utility **** --init [TEMPLATE...] Initialize your project with ShellSpec | [spec] [git] [hg] [svn] --gen-bin [@COMMAND...] Generate test support commands in "<HELPERDIR>/support/bin" --count Count the number of specfiles and examples --list LIST List the specfiles/examples | [specfiles] [examples(:id|:lineno)] --syntax-check Syntax check of the specfiles without running any examples --translate Output translated specfile --task [TASK] Run the TASK or Show the task list if TASK is not specified --docker DOCKER-IMAGE Run tests in specified docker image (EXPERIMENTAL) -v, --version Display the version -h, --help -h: short help, --help: long help
All specfiles for ShellSpec must be under the project directory. The root of the project directorymust have a.shellspec
file. This file is that specify the default options to be used inthe project, but an empty file is required even if the project has no options.
NOTE: The.shellspec
file was described in the documentation as a required file for some time,but ShellSpec worked without it. Starting with version 0.28.0, this file is checked and will berequired in future versions.
You can easily create the necessary files by executing theshellspec --init
command in an existing directory.
This is the typical directory structure. Version 0.28.0 allows many of these to be changed by specifying options, supporting a more flexibledirectory structure.
<PROJECT-ROOT> directory├─ .shellspec [mandatory]├─ .shellspec-local [optional] Ignore from version control├─ .shellspec-quick.log [optional] Ignore from version control├─ report/ [optional] Ignore from version control├─ coverage/ [optional] Ignore from version control│├─ bin/│ ├─ your_script1.sh│ :├─ lib/│ ├─ your_library1.sh│ :│├─ spec/ (also <HELPERDIR>)│ ├─ spec_helper.sh [recommended]│ ├─ banner[.md] [optional]│ ├─ support/ [optional]│ ││ ├─ bin/│ │ ├─ your_script1_spec.sh│ │ :│ ├─ lib/│ │ ├─ your_library1_spec.sh
To change the default options for theshellspec
command, create options file(s).Files are read in the order shown below, options defined last take precedence.
$XDG_CONFIG_HOME/shellspec/options
$HOME/.shellspec-options
(version >= 0.28.0) or$HOME/.shellspec
(deprecated)<PROJECT-ROOT>/.shellspec
<PROJECT-ROOT>/.shellspec-local
(Do not store in VCS such as git)
Specify your default options with$XDG_CONFIG_HOME/shellspec/options
or$HOME/.shellspec-options
.Specify default project options with.shellspec
and overwrite to your favorites with.shellspec-local
.
Specifies the default options to use for the project.
Override the default options used by the project with your favorites.
Used to specify the directory in which the specfile will be run.Seedirectory structure or--execdir
option for details.
If this file is present, Quick mode will be enabled and the log of Quick execution will be recorded.It created automatically when--quick
option is specified.If you want to turn off Quick mode, delete it.
The output location for reports generated by the--output
or--profile
options.This can be changed with the--reportdir
option.
The output location for coverage reports.This can be changed with the--covdir
option.
By default, it is assumed that all specfiles are store under thespec
directory,but it is possible to create multiple directories with different names.
NOTE: In Version <= 0.27.x, thespec
directory was the only directory that contained the specfiles.
The directory to storespec_helper.sh
and other files.By default, thespec
directory also serves asHELPERDIR
directory,but you can change it to another directory with the--helperdir
option.
Thespec_helper.sh
is loaded to specfile by the--require spec_helper
option.This file is used to define global functions, initial setting for examples, custom matchers, etc.
If the file<HELPERDIR>/banner
or<HELPERDIR>/banner.md
exists, Display a banner whentheshellspec
command is executed. It can be used to display information about the tests.The--no-banner
option can be used to disable this behavior.
This directory can be used to store files such as custom matchers and tasks.
This directory is used to storesupport commands.
In ShellSpec, you write your tests in a specfile.By default, specfile is a file ending with_spec.sh
under thespec
directory.
The specfile is executed using theshellspec
command, but it can also be executed directly.Seeself-executable specfile for details.
Describe'lib.sh'# example group Describe'bc command'add() {echo"$1 +$2"| bc; } It'performs addition'# example When call add 2 3# evaluation The output should eq 5# expectation End EndEnd
The best place to learn how to write a specfile is theexamples/spec directory. You should take a look at it!(Those examples include failure examples on purpose.)
ShellSpec has its own DSL to write tests. It may seem like a distinctive code because DSL startswith a capital letter, but the syntax is compatible with shell scripts, and you can embedshell functions and useShellCheck to check the syntax.
You may feel rejected by this DSL, but It starts with a capital letter to avoid confusion withthe command, and it does a lot more than you think, such as realizing scopes, gettingshell-independent line numbers, and workarounds for bugs in some shells.
Since version 0.28.0, the working directory when running a specfile is the project root directory by default.Even if you run a specfile from a subdirectory in the project directory,it is the project root directory.Before 0.27.x, it was the current directory when theshellspec
command was executed.
You can change this directory (location) by using the--execdir @LOCATION[/DIR]
option.You can choose from the following locations and specify a path relative to the location if necessary.However, you cannot specify a directory outside the project directory.
@project
Where the.shellspec
file is located (project root) [default]@basedir
Where the.shellspec
or.shellspec-basedir
file is located@specfile
Where the specfile is located
If@basedir
is specified, the parent directory is searched from the directory containing the specfileto be run, and the first directory where.shellspec-basedir
or.shellspec
is found is used asthe execution directory. This is useful if you want to have a separate directory for eachutility (command) you want to test.
NOTE: You will need to change under the project directory or use the-c
(--chdir
) or-C
(--directory
) option before running the specfile.
You can embed shell functions (or shell script code) in the specfile.This shell function can be used for test preparation and complex testing.
Note that the specfile implements scope using subshells.Shell functions defined in the specfile can only be used within blocks (e.g.Describe
,It
, etc).
If you want to use a global function, you can define it inspec_helper.sh
.
The specfile will not be executed directly by the shell, but will be translated into a regularshell script and output to a temporary directory (default:/tmp
) before being executed.
The translation process is simple in that it only replaces forward-matched words (DSLs), with a fewexceptions. If you are interested in the translated code, you can see withshellspec --translate
.
The specfile contains DSLs, so it cannot be properly formatted by general shell script formatters.If you want to format the specfile, usealtshfmt.
ExampleGroup
is a block for grouping example groups or examples.Describe
andContext
are aliases forExampleGroup
.They can be nested, and may contain example groups or examples.
Describe'is example group' Describe'is nestable' ... End Context'is used to facilitate understanding depending on the context' ... EndEnd
The example groups can be optionally tagged. SeeTagging for details.
Describe'is example group' tag1:value1 tag2:value2 ...
Example
is a block for writing an evaluation and expectations.It
andSpecify
are aliases forExample
.
An example is composed by up to one evaluation and multiple expectations.
add() {echo"$1 +$2"| bc; }It'performs addition'# example When call add 2 3# evaluation The output should eq 5# expectation The status should be success# another expectationEnd
The examples can be optionally tagged. SeeTagging for details.
It'performs addition' tag1:value1 tag2:value2 ...
Todo
is the same as the empty example and is treated as apending example.
Todo'will be used later when we write a test'It'is an empty example, the same as Todo'End
Evaluation executes a shell function or command for verification.Only one evaluation can be defined for each example and also can be omitted.
See more details ofEvaluation
It calls a function without subshell.Practically, it can also run commands.
When call add 1 2# call `add` shell function with two arguments.
It runs a command within subshell. Practically, it can also call a shell function.The command does not have to be a shell script.
NOTE: This does not support coverage measurement.
When run touch /tmp/foo# run `touch` command.
Some commands below are specially handled by ShellSpec.
It runs a command, respecting shebang.It can not call a shell function. The command does not have to be a shell script.
NOTE: This does not support coverage measurement.
When runcommand touch /tmp/foo# run `touch` command.
It runs a shell script, ignoring shebang. The script has to be a shell script.It will be executed in another instance of the same shell as the current shell.
When run script my.sh# run `my.sh` script.
It sources a shell script, ignoring its shebang. The script has to be a shell script.It is similar torun script
, but with some differences.Unlikerun script
, function-based mocking is available.
When runsource my.sh# source `my.sh` script.
If you want to execute aliases, you need a workaround usingeval
.
alias alias-name='echo this is alias'When call alias-name# alias-name: not found# eval is requiredWhen calleval alias-name# When using embedded shell scriptsfoo() {eval alias-name; }When call foo
Expectation begins withThe
which does the verification.The basic syntax is as follows:
The output should equal 4
Useshould not
for the opposite verification.
The output should not equal 4
The subject is the target of the verification.
The output should equal 4| +-- subject
There areoutput
(stdout
),error
(stdout
),status
,variable
,path
, etc. subjects.
Please refer to theSubjects for more details.
The modifier concretizes the target of the verification (subject).
The line 2 of output should equal 4| +-- modifier
The modifiers are chainable.
The word 1 of line 2 of output should equal 4
If the modifier argument is a number, you can use an ordinal numeral instead of a number.
The first word of second line of output should equal 4
There areline
,word
,length
,contents
,result
, etc. modifiers.Theresult
modifier is useful for making the result of a user-defined function the subject.
Please refer to theModifiers for more details.
The matcher is the verification.
The output should equal 4| +-- matcher
There are many matchers such as string matcher, status matcher, variable matchers and stat matchers.Thesatisfy
matcher is useful for verification with user-defined functions.
Please refer to theMatchers for more details.
ShellSpec supportslanguage chains likechai.js.It only improves readability, does not affect the expectation:a
,an
,as
,the
.
The following two sentences have the same meaning:
The first word of second line of output should valid numberThe first word of the second line of output should valid as a number
TheAssert
is yet another expectation to verify with a user-defined function.It is designed for verification of side effects, not the result of the evaluation.
still_alive() { ping -c1"$1">/dev/null}Describe"example.com" It"responses" Assert still_alive"example.com" EndEnd
Pending
is similar toSkip
, but the test passes if the verification fails,and the test fails if the verification succeeds. This is useful if you want tospecify that you will implement something later.
Describe'Pending' Pending"not implemented"hello() {:; } It'will success when test fails' When call hello world The output should"Hello world" EndEnd
UseSkip
to skip executing the example.
Describe'Skip' Skip"not exists bc" It'is always skip' ... EndEnd
UseSkip if
if you want to skip conditionally.
Describe'Conditional skip'not_exists_bc() {!type bc>/dev/null2>&1; } Skipif"not exists bc" not_exists_bcadd() {echo"$1 +$2"| bc; } It'performs addition' When call add 2 3 The output should eq 5 EndEnd
xDescribe
,xContext
,xExampleGroup
are skipped example group blocks.Execution of examples contained in these blocks is skipped.
Describe'is example group' xDescribe'is skipped example group' ... EndEnd
xIt
,xSpecify
,xExample
are skipped example blocks.Execution of the example is skipped.
xIt'is skipped example' ...End
fDescribe
,fContext
,fExampleGroup
are focused example group blocks.Only the examples included in these will be executed when the--focus
option is specified.
Describe'is example group' fDescribe'is focus example group' ... EndEnd
fIt
,fSpecify
,fExample
are focused example blocks.Only these examples will be executed when the--focus
option is specified.
fIt'is focused example' ...End
UsingPending
orSkip
without a message is a "temporary pending" or "temporary skip"."x"-prefixed example groups and examples are also treated as temporary skips.
The non-temporaryPending
andSkip
(with a message) are used when the case will take a long time to resolve.It may be committed to a version control system. Temporary pending and skip are used during current work.We do not recommend committing them to a version control system.
These two types differ in the display of the report. Refer to--skip-message
and--pending-message
options.
# Temporary pending and skipPendingSkipSkip# this comment will be displayed in the reportTodoxIt ...End# Non-temporary pending and skipPending"reason"Skip"reason"Skipif"reason" conditionTodo"It will be implemented"
You can specify commands to be executed before / after each example byBeforeEach
(Before
),AfterEach
(After
).
NOTE:BeforeEach
andAfterEach
are supported in version 0.28.0 and later.Previous versions should useBefore
andAfter
instead.
NOTE:AfterEach
is for cleanup and not for assertions.
Describe'example hook'setup() {:; }cleanup() {:; } BeforeEach'setup' AfterEach'cleanup' It'is called before and after each example' ... End It'is called before and after each example' ... EndEnd
You can specify commands to be executed before / after all examples byBeforeAll
andAfterAll
.
Describe'example all hook'setup() {:; }cleanup() {:; } BeforeAll'setup' AfterAll'cleanup' It'is called before/after all example' ... End It'is called before/after all example' ... EndEnd
You can specify commands to be executed before / after call evaluation byBeforeCall
andAfterCall
.
NOTE: These hooks were originally created to test ShellSpec itself.Please use theBeforeEach
/AfterEach
hooks whenever possible.
Describe'call evaluation hook'setup() {:; }cleanup() {:; } BeforeCall'setup' AfterCall'cleanup' It'is called before/after call evaluation' When call hello world ... EndEnd
You can specify commands to be executed before / after run evaluation(run
,run command
,run script
, andrun source
) byBeforeRun
andAfterRun
.
These hooks are executed in the same subshell as the "run evaluation".Therefore, you can access the variables after executing the evaluation.
NOTE: These hooks were originally created to test ShellSpec itself.Please use theBeforeEach
/AfterEach
hooks whenever possible.
Describe'run evaluation hook'setup() {:; }cleanup() {:; } BeforeRun'setup' AfterRun'cleanup' It'is called before/after run evaluation' When run hello world ... EndEnd
The hooks may fail in subtle ways if there is output to stderr, even if thereturn code / exit code is0
.
Commands likegit checkout
routinely write to stderr, even if there was no actualfailure, so be aware that your hooks may fail because of this.
Dump stdout, stderr, and status of the evaluation. It is useful for debugging.
When callecho hello worldDump# stdout, stderr and status
Include a shell script to test.
Describe'lib.sh' Include lib.sh# hello function defined Describe'hello()' It'says hello' When call hello ShellSpec The output should equal'Hello ShellSpec!' End EndEnd
Set shell options before executing each example.The shell option name is the long name ofset
or the name ofshopt
:
NOTE: UseSet
instead of theset
command because theset
commandmay not work as expected in some shells.
Describe'Set helper' Set'errexit:off''noglob:on' It'sets shell options before executing the example' When call foo EndEnd
Path
is used to define a short pathname alias.File
andDir
are aliases forPath
.
Describe'Path helper' Path hosts-file="/etc/hosts" It'defines short alias for long path' The path hosts-file should exist EndEnd
You can use the Data Helper which inputs data from stdin for evaluation.The input data is specified after#|
in theData
orData:expand
block.
Describe'Data helper' It'provides with Data helper block style' Data# Use Data:expand instead if you want expand variables.#|item1 123#|item2 456#|item3 789 End When call awk'{total+=$2} END{print total}' The output should eq 1368 EndEnd
You can also use a file, function or string as data sources.
See more details ofData
Parameterized tests (akaData-driven testing)are used to run the same test once for each set of parameters.Parameters
defines a block of parameters.
Describe'example' Parameters"#1" 1 2 3"#2" 1 2 3 End Example"example$1" When callecho"$(($2+$3))" The output should eq"$4" EndEnd
In addition to the defaultParameters
block, three additional styles are supported:Parameters:value
,Parameters:matrix
andParameters:dynamic
.
See more details ofParameters
NOTE: You can also combine theParameters
andData:expand
helpers.
SeeIntercept
Directives are instructions that can be used in embedded shell scripts.It is used to solve small problems of shell scripts in testing.
This is like a shell function, but not a shell function.Therefore, the supported grammar is limited and can only be used at thebeginning of a function definition or at the beginning of a line.
foo() { %puts"foo"; }# supportedbar() { %puts"bar"# supported}baz() { anycommand; %puts"baz"# not supported}
%const
(%
is short hand) directive defines a constant value. The characterswhich can be used for variable names are uppercase letters[A-Z]
, digits[0-9]
and underscore_
only. It can not be defined inside an examplegroup nor an example.
The value is evaluated during the specfile translation process.So you can access ShellSpec variables, but you can not access variable orfunction in the specfile.
This feature assumes use with conditional skip. The conditional skip may runoutside of the examples. As a result, sometimes you may need variables definedoutside of the examples.
You can use the%text
directive instead of a hard-to-use heredoc withindented code. The input data is specified after#|
.
Describe'%text directive' It'outputs texts'output() {echo"start"# you can write code here %text#|aaa#|bbb#|cccecho"end"# you can write code here }result() { %text#|start#|aaa#|bbb#|ccc#|end } When call output The output should eq"$(result)" The line 3 of output should eq'bbb' EndEnd
%puts
(put string) and%putsn
(put string with newline) can be used insteadof (not portable) echo. Unlike echo, it does not interpret escape sequencesregardless of the shell.%-
is an alias of%puts
,%=
is an alias of%putsn
.
This is the same asprintf
, but it can be used in the sandbox mode because the path has been resolved.
This is the same assleep
, but it can be used in the sandbox mode because the path has been resolved.
Use the%preserve
directive to preserve the variables in subshells and external shell scripts.
In the following cases,%preserve
is required because variables are not preserved.
When run
evaluation - It runs in a subshell.- Command-based mock (
Mock
) - It is an external shell script. - Function-based Mock called by command substitution
Describe'%preserve directive' It'preserves variables'func() { foo=1; bar=2; baz=3; }preserve() { %preserve bar baz:BAZ; } AfterRun preserve When run func The variable foo should eq 1# This will be failure The variable bar should eq 2# This will be success The variable BAZ should eq 3# Preserved to different variable (baz:BAZ) EndEnd
Output log messages to the log file (default:/dev/tty
) for debugging.
There are two ways to create a mock, (shell) function-based mock and (external) command-based mock.The function-based mock is usually recommended for performance reasons.Both can be overwritten with an internal block and will be restored when the block ends.
The (shell) function-based mock is simply (re)defined with a shell function.
Describe'function-based mock'get_next_day() {echo$(($(date+%s)+86400)); }date() {echo 1546268400 } It'calls the date function' When call get_next_day The stdout should eq 1546354800 EndEnd
The (external) command-based mock creates a temporary mock shell script and runs as an external command.This is slow, but there are some advantages over the function-based mock.
- Can be use invalid characters as the shell function name.
- e.g.
docker-compose
(-
cannot be used as a function name in POSIX)
- e.g.
- Can be invoke a mocked command from an external command (not limited to shell script).
A command-based mock creates an external shell script with the contents of aMock
block,so there are some restrictions.
- It is not possible to mock shell functions or shell built-in functions.
- It is not possible to call shell functions outside the
Mock
block.- Exception: Can be called exported (
export -f
) functions. (bash only)
- Exception: Can be called exported (
- To reference variables outside the
Mock
block, they must be exported. - To return a variable from a Mock block, you need to use the
%preserve
directive.
Describe'command-based mock'get_next_day() {echo$(($(date+%s)+86400)); } Mock dateecho 1546268400 End It'runs the mocked date command' When call get_next_day The stdout should eq 1546354800 EndEnd
NOTE: To achieve this feature, a directory for mock commands is included at the beginning of thePATH
.
Support commands are helper commands that can be used in the specfile.For example, it can be used in a mock function to execute the actual command.It is recommended that the support command name be the actual command name prefixed with@
.
Describe"Support commands example"touch() { @touch"$@"# @touch executes actual touch commandecho"$1 was touched" } It"touch a file" When run touch"file" The output should eq"file was touched" The file"file" should exist EndEnd
Support commands are generated in thespec/support/bin
directory by the--gen-bin
option.For example runshellspec --gen-bin @touch
to generate the@touch
command.
This is the main purpose, but support commands are just shell scripts, so they canalso be used for other purposes. You can freely edit the support command script.
The sandbox mode forces the use of mocks. However, you may not want to require mocks for some commands.For example,printf
is a built-in command in many shells and does not require a mock in the sandbox mode for these shells. Butthere are shells where it is an external command and then it requires to be mocked.
To allowprintf
to be called without mocking in certain cases,create a support command namedprintf
(shellspec --gen-bin printf
).
Some commands have different options between BSD and GNU.If you handle the difference in the specfile, the test will be hard to read.You can solve it with the support command.
#!/bin/sh -e# Command name: @sed."$SHELLSPEC_SUPPORT_BIN"case$OSTYPEin*darwin*) invoke gsed"$@" ;;*) invoke sed"$@" ;;esac
The example groups or examples can be tagged, and the--tag
option can be used to filter the examples to be run.The tag name and tag value are separated by:
, and the tag value is optional. You can use any character if quoted.
Describe"Checking something" someTag:someVal It"does foo" tagA:val1 ... It"does bar" tagA:val2 ... It"does baz" tagA ...End
- Everything nested inside a selected element is selected in parent elements. e.g.
--tag someTag
will select everything above. - Specifying a tag but no value selects everything with that tag whether or not it has a value, e.g.
--tag tagA
will select everything above. - Specifying multiple tags will select the union of everything tagged, e.g.
--tag tagA:val1,tagA:val2
will selectdoes foo
anddoes bar
. - Tests included multiple times are not a problem, e.g.
--tag someTag,tagA,tagA:val1
just selects everything. - If no tag matches, nothing will be run, e.g.
--tag tagA:
runs nothing (it does not match baz above, as empty values are not the same as no value). - The --tag option can be used multiple times, e.g.
--tag tagA:val1 --tag tagA:val2
works the same as--tag tagA:val1,tagA:val2
ShellSpec is a testing framework for shell scripts, but it can be used to test anything that can be executed as an external command, even if it is written in another language. Even shell scripts can be tested as external commands.
If you are testing a shell script as an external command, please note the following.
- It will be executed in the shell specified by the shebang not the shell running the specfile.
- The coverage of the shell script will not be measured.
- Cannot refer to variables inside the shell script.
- Shell built-in commands cannot be mocked.
- Functions defined inside the shell script cannot be mocked.
- Only command-based mock can be used (if the script is calling an external command).
- Interceptor is not available.
To get around these limitations, userun script
orrun source
. SeeHow to test a single file shell script.
If the shell script consists of a single file, unit testing becomes difficult.However, there are many such shell scripts.
ShellSpec has the ability to testing in such cases with only few modifications to the shell script.
Unlike the case of executing as anexternal command, it has the following features.
- It will run in the same shell (but another process) that is running specfile.
- The coverage of the shell script will be measured.
There are limitations as follows.
- Cannot refer to variables inside the shell script.
- Shell built-in commands cannot be mocked.
- Functions defined inside the shell script cannot be mocked.
- Only command-based mock can be used (if the script is calling an external command).
- Interceptor is not available.
It is even less limitations thanrun script
and has the following features.
- It will run in the same shell and same process that is running specfile.
- The coverage of the shell script will be measured.
- Can be refer to variables inside the shell script.
- Function-based mock and command-based mock are available.
- Interceptor is available.
- Shell built-in commands can be mocked.
- Functions defined inside the shell script can be mocked using interceptor.
However, since it is simulated using the.
command, there are some differences in behavior.For example, the value of$0
is different.
NOTE: Mocking of shell built-in commands can be done beforerun source
. However, if you are usinginterceptor, mocking of thetest
command must be done in the__<name>__
function.
This is the way to test shell functions defined in a shell script.
Loading a script withInclude
defines a__SOURCED__
variable available in the sourced script.If the variable__SOURCED__
is defined, please return from the shell script.
#!/bin/sh# hello.shhello() {echo"Hello$1"; }# This is the writing style presented by ShellSpec, which is short but unfamiliar.# Note that it returns the current exit status (could be non-zero).${__SOURCED__:+return}# The above means the same as below.# ${__SOURCED__:+x} && return $?# If you don't like the coding style, you can use the general writing style.# if [ "${__SOURCED__:+x}" ]; then# return 0# fihello"$1"
Describe"hello.sh" Include"./hello.sh" Describe"hello()" It"says hello" When call hello world The output should eq"Hello world" End EndEnd
Interceptor is a feature that allows you to intercept your shell script in the middle of its execution.This makes it possible to mock functions that cannot be mocked in advance at arbitrary timing,and to make assertions by retrieving the state of during script execution.
It is a powerful feature, but avoid using it as possible, because it requires you to modify your codeand may reduce readability. Normally, it is not a good idea to modify the code just for testing,but in some cases, there is no choice but to use this.
#!/bin/sh# ./today.sh# When run directly without testing, the "__()" function does nothing.test||__() {:; }# the "now()" function is defined here, so it can't be mocked in advance.now() { date +"%Y-%m-%d %H:%M:%S"; }# The function you want to testtoday() { now=$(now)echo"${now%*}"}# I want to mock the "now()" function here.__ begin __today=$(today)echo"Today is$today"__ end __
Describe"today.sh" Intercept begin__begin__() {now() {echo"2021-01-01 01:02:03"; } }__end__() {# The "run source" is run in a subshell, so you need to use "%preserve"# to preserve variables %preserve today } It"gets today's date" When runsource ./today.sh The output should eq"Today is 2021-01-01" The variable today should eq"2021-01-01" EndEnd
Usage:Intercept [<name>...]
Specify the name(s) to intercept.
NOTE: I will changeIntercept
toInterceptors
to make it a declarative DSL.
Define the__
function that does nothing except when run as a test (via ShellSpec).This allows you to run it as a production without changing the code.
Thetest
command is the shell built-intest
command. This command returns false (non-zero)when called with no arguments. This will allow who are not familiar with ShellSpec to willunderstand what the result will be, even if they don't know what the code is for.Of course, it is good practice to comment on what the code is for
When run via ShellSpec, thetest
command is redefined and returns true "only once" when calledwith no arguments. After that, it will return to its original behavior. This means that this codeneeds to be executed only once, at the start of the shell script.
Usage:__ <name> [arguments...] __
This is where the process is intercepted. You can define more than one.If the name matches the name specified inIntercept
, the__<name>__
function will be called.
Note that if the name is not specified inIntercept
, nothing will be done,but the exit status will be changed to 0.
Thespec_helper
can be used to set shell options for all specfiles,define global functions,check the execution shell, load custom matchers, etc.
Thespec_helper
is the default module name. It can be changed to any other name, and multiplemodules can be used. Only characters accepted by POSIX as identifiers can be used in module names.The file name of the module must be the module name with the extension.sh
appended.It is loaded fromSHELLSPEC_LOAD_PATH
using the--require
option.
The following is a typicalspec_helper
. The following three callback functions are available.
# Filename: spec/spec_helper.shset -euspec_helper_precheck() { minimum_version"0.28.0"if ["$SHELL_TYPE"!="bash" ];then abort"Only bash is supported."fi}spec_helper_loaded() {:# In most cases, you won't use it.}spec_helper_configure() { import'support/custom_matcher' before_each"global_before_each_hook"}# User-defined global functionglobal_before_each_hook() {:}# In version <= 0.27.x, only shellspec_spec_helper_configure was available.# This callback function is still supported but deprecated in the future.# Please rename it to spec_helper_configure.# shellspec_spec_helper_configure() {# :# }
Thespec_helper
will be loaded at least twice. The first time is at precheck phase,which is executed in a separate process before the specfile execution.The second time will be load at the beginning of the specfile execution.If you are using parallel execution, it will be loaded every specfile.
Within each callback function, there are several helper functions available. These functions arenot available outside of the callback function. Also, these callback functions will be removedautomatically whenspec_helper
is finished loading. (User-defined functions will not be removed.)
This callback function will be invoked only once before loading specfiles.Exit withexit
orabort
, orreturn
non-zero to exit without executing specfiles.Inside this function,set -eu
is executed, so an explicit return on error is not necessary.
Since it is invoked in a separate process from specfiles, changes made inthis function will not be affected in specfiles.
- Usage:
minimum_version <version>
Specifies the minimum version of ShellSpec that the specfile supports. The version format issemantic version. Pre-release versions have a lower precedence thanthe associated normal version, but comparison between pre-release versions is not supported.The build metadata will simply be ignored.
NOTE: Since<module>_precheck
is only available in 0.28.0 or later,it can be executed with earlier ShellSpecs even if minimum_version is specified.To avoid this, you can implement a workaround using--env-from
.
# spec/env.sh# Add `--env-from spec/env.sh` to `.shellspec`major_minor=${SHELLSPEC_VERSION%".${SHELLSPEC_VERSION#*.*.}"}if ["${major_minor%.*}"-eq 0 ]&& ["${major_minor#*.}"-lt 28 ];thenecho"ShellSpec version 0.28.0 or higher is required.">&2exit 1fi
- Usage:
error [messages...]
- Usage:
warn [messages...]
- Usage:
info [messages...]
Outputs a message according to the type. You can also useecho
orprintf
.
- Usage:
abort [messages...]
- Usage:
abort <exit status> [messages...]
Display an error message andexit
. If the exit status is omitted, it is1
.You can also exit with exit.exit 0
will exit normally without executing the specfiles.
- Usage:
setenv [name=value...]
- Usage:
unset [name...]
You can usesetenv
orunsetenv
to pass or remove environment variables from precheck to specfiles.
The following environment variables are defined.
VERSION
- ShellSpec VersionSHELL_TYPE
- Currently running shell type (e.g.bash
)SHELL_VERSION
- Currently running shell version (e.g.4.4.20(1)-release
)
NOTE: Be careful not to confuseSHELL_TYPE
with the environment variableSHELL
.The environment variableSHELL
is the user login shell, not the currently running shell.It is a variable set by the system, and which unrelated to ShellSpec.
It is called after loading the shellspec's general internal functions,but before loading the core modules (subject, modifier, matcher, etc).If parallel execution is enabled, it may be called multiple times in isolated processes.Internal functions starting withshellspec_
can also be used, but be aware that they may change.
This was created to performworkarounds for specific shells in order totest ShellSpec itself. Other than that, I have not come up with a case where this isabsolutely necessary, but if you have one, please let me know.
This callback function will be called after core modules (subject, modifier, matcher, etc) has been loaded.If parallel execution is enabled, it may be called multiple times in isolated processes.Internal functions starting withshellspec_
can also be used, but be aware that they may change.It can be used to set global hooks, load custom matchers, etc., and override core module functions.
- Usage:
import <module> [arguments...]
Import a custom module fromSHELLSPEC_LOAD_PATH
.
- Usage:
before_each [hooks...]
- Usage:
after_each [hooks...]
Register hooks to be executed before and after every example.It is the same as executingBeforeEach
/AfterEach
at the top of all specfiles.
- Usage:
before_all [hooks...]
- Usage:
after_all [hooks...]
Register hooks to be executed before and after all example.It is the same as executingBeforeAll
/AfterAll
at the top of all specfiles.
NOTE: This is a hook that is called before and after each specfile, not before and after all specfiles.
Addeval "$(shellspec - -c) exit 1"
to the top of the specfile and give execute permissionto the specfile. You can use/bin/sh
,/usr/bin/env bash
, etc. for shebang.The specfile will be run in the shell written in shebang.
#!/bin/sheval"$(shellspec - -c) exit 1"# Use the following if version <= 0.27.x# eval "$(shellspec -)"Describe"bc command"bc() {echo"$@"|command bc; } It"performs addition" When call bc"2+3" The output should eq 5 EndEnd
The-c
option is available since 0.28.0, and you can also pass other options.If you run the specfile directly,--pattern
will be automatically set to*
.These options will be ignored if run viashellspec
command.
The use ofshellspec
as shebang is deprecated because it is not portable.
#!/usr/bin/env shellspec -cLinux does not allow passing options#!/usr/bin/env -S shellspec -cThe-S option requires GNU Core Utilities8.30 (2018-07-01) or later.
You can run ShellSpec without installation using Docker. ShellSpec andspecfiles run in a Docker container.
SeeHow to use ShellSpec with Docker.
You can create custom subject, custom modifier and custom matcher.
Seeexamples/spec/support/custom_matcher.sh for custom matcher.
NOTE: If you want to verify using shell function, you can useresult modifier orsatisfy matcher. You don't need to create a custom matcher, etc.
ShellSpec has integrated coverage feature. To use this featureKcov (v38 or later) is required.
Supported only inbash,zsh, andksh, whereDEBUG
trap is implemented.However, we recommend latestbash for the following reasons.
- bash (older versions): There seems to be some code that cannot be measured correctly.
- zsh: There seems to be some code that cannot be measured correctly, and the measurement rate will be the lowest.
- ksh: ksh93u+ and ksh2020 may have side effects on exit status when DEBUG trap is enabled due to a bug.This bug has beenfixed inksh93u+m.These are also prone to instability, especially with ksh2020 (which has been abandoned).
In any shell, some code may not be measured correctly (e.g., code containingeval
or newline).These are limitations and problems caused by shells and Kcov.
ShellSpec measures only the necessary codes to improve the measurement speed.Also, there are some things that cannot be measured due to implementation.
- The shell scripts loaded by
Include
will be measured. - The shell functions called by the
When
evaluation will be measured. - The shell scripts executed by the
When run script
evaluation will be measured. - The shell scripts executed by the
When run source
evaluation will be measured. - The external commands executed by the
When
evaluation will NOT be measured.- Even if it is a shell script, it is not measured when it is executed as an external command.
- If other than the above, it will not be measured.
By default only shell scripts whose names contain.sh
are coverage targeted.If you want to include other files, you need to adjust options with--kcov-options
.
# Default kcov (coverage) options--kcov-options"--include-path=. --path-strip-level=1"--kcov-options"--include-pattern=.sh"--kcov-options"--exclude-pattern=/.shellspec,/spec/,/coverage/,/report/"# Example: Include script "myprog" with no extension--kcov-options"--include-pattern=.sh,myprog"# Example: Only specified files/directories--kcov-options"--include-pattern=myprog,/lib/"
[Coverage report][coverage] andcobertura.xml
andsonarqube.xml
files are generated under the coverage directory by Kcov.You can easily integrate withCoveralls,Code Climate,Codecov, etc.
URL:https://github.com/shellspec/shellmetrics
URL:https://github.com/shellspec/shellbench
URL:https://github.com/shellspec/altshfmt
URL:https://github.com/ko1nksm/getoptions
URL:https://github.com/ko1nksm/readlinkf
URL:https://github.com/ko1nksm/portable-echo.sh
- RSpec - Behaviour Driven Development for Ruby
- Jest - Delightful JavaScript Testing
- Mocha - the fun, simple, flexible JavaScript test framework
- Jasmine - Behavior-Driven JavaScript
- Ginkgo - A Golang BDD Testing Framework
- JUnit 5 - The programmer-friendly testing framework for Java
All contributions are welcome!
ShellSpec uses a peculiar coding style to assure high performance,reliability and portability, and the external commands allowed to use are greatly restricted.
We recommend that you create WIP PR early or offer suggestions in discussions to avoid ruining your work.
Author: Koichi Nakashima (ko1nksm)
About
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
Topics
Resources
License
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Sponsor this project
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.