Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork56
bash unit testing enterprise edition framework for professionals
License
bash-unit/bash_unit
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
bash_unit - bash unit testing enterprise edition framework for professionals!
bash_unit allows you to write unit tests (functions starting withtest),run them and, in case of failure, displays the stack tracewith source file and line number indications to locate the problem.
Need a quick start? Thegetting started projectwill help you get on track in no time.
The following functions are available in your tests (see below for detailed documentation):
fail [message]assert <assertion> [message]assert_fail <assertion> [message]assert_status_code <expected_status_code> <assertion> [message]assert_equals <expected> <actual> [message]assert_not_equals <unexpected> <actual> [message]assert_matches <expected-regex> <actual> [message]assert_not_matches <unexpected-regex> <actual> [message]assert_within_delta <expected num> <actual num> <max delta> [message]assert_no_diff <expected> <actual> [message]skip_if <condition> <pattern>fake <command> [replacement code]
(by the way, the documentation you are reading is itself tested with bash-unit)
bash_unit is free software you may contribute to. SeeCONTRIBUTING.md.
- -ppattern
filters tests to run based on the given pattern.You can specify several patterns by repeating this optionfor each pattern.
- -spattern
skip tests which name matches the given pattern.You can specify several patterns by repeating this optionfor each pattern.Tests will appear inbash_unit output asskipped.(see alsoskip_if)
- -r
executes test cases in random order.Only affects the order within a test file (files are alwaysexecuted in the order in which they are specified on thecommand line).
- -foutput_format
specify an alternative output format.The only supported value istap.
- -q
quiet mode.Will only output the status of each test with no furtherinformation even in case of failure.
bash_unit package is available on Archlinux through AUR. In order to install, issue the following command :
yaourt -Sys bash_unit
installing viaNix/NixOS
bash_unit package has been added tonixpkgs. You can use it with the following command:
nix-shell -p bash_unit
This will installbash_unit in your current working directory:
curl -s https://raw.githubusercontent.com/bash-unit/bash_unit/master/install.sh | bash
You can also download it from therelease page.
Here is an example of how you could integratebash_unit withGitHub Actions:
name: bash_unit testson: push: branches: [ main ] pull_request: branches: [ main ]jobs: ubuntu: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Unit testing with bash_unit run: | curl -s https://raw.githubusercontent.com/bash-unit/bash_unit/master/install.sh | bash FORCE_COLOR=true ./bash_unit tests/test_*See this bash_unitgetting started github project for a working example.
Here is an example of how you could integratebash_unit withGitLab CI:
test: image: debian script: - apt-get update - apt-get install --no-install-recommends -y curl ca-certificates - curl -s https://raw.githubusercontent.com/bash-unit/bash_unit/master/install.sh | bash - FORCE_COLOR=true ./bash_unit tests/test_*See this bash_unitgetting started gitlab project for a working example.
You can runbash_unit as apre-commit hook.
Add the following to your pre-commit configuration. By default it will run scripts that are identified as shell scripts that match the path^tests/(.*/)?test_.*\.sh$.
repos: - repo: https://github.com/bash-unit/bash_unit rev: v2.2.0 hooks: - id: bash-unit always-run: trueTo run tests, simply callbash_unit with all your tests files as parameter. For instance to run somebash_unit tests, frombash_unit directory:
./bash_unit tests/test_core.shRunning tests in tests/test_core.shRunning test_assert_equals_fails_when_not_equal ... SUCCESSRunning test_assert_equals_succeed_when_equal ... SUCCESSRunning test_assert_fails ... SUCCESSRunning test_assert_fails_fails ... SUCCESSRunning test_assert_fails_succeeds ... SUCCESSRunning test_assert_matches_fails_when_not_matching ... SUCCESSRunning test_assert_matches_succeed_when_matching ... SUCCESSRunning test_assert_no_diff_fails_when_diff ... SUCCESSRunning test_assert_no_diff_succeeds_when_no_diff ... SUCCESSRunning test_assert_not_equals_fails_when_equal ... SUCCESSRunning test_assert_not_equals_succeeds_when_not_equal ... SUCCESSRunning test_assert_not_matches_fails_when_matching ... SUCCESSRunning test_assert_not_matches_succeed_when_not_matching ... SUCCESSRunning test_assert_shows_stderr_on_failure ... SUCCESSRunning test_assert_shows_stdout_on_failure ... SUCCESSRunning test_assert_status_code_fails ... SUCCESSRunning test_assert_status_code_succeeds ... SUCCESSRunning test_assert_succeeds ... SUCCESSRunning test_assert_within_delta_fails ... SUCCESSRunning test_assert_within_delta_succeeds ... SUCCESSRunning test_fail_fails ... SUCCESSRunning test_fail_prints_failure_message ... SUCCESSRunning test_fail_prints_where_is_error ... SUCCESSRunning test_fake_actually_fakes_the_command ... SUCCESSRunning test_fake_can_fake_inline ... SUCCESSRunning test_fake_echo_stdin_when_no_params ... SUCCESSRunning test_fake_exports_faked_in_subshells ... SUCCESSRunning test_fake_transmits_params_to_fake_code ... SUCCESSRunning test_fake_transmits_params_to_fake_code_as_array ... SUCCESSRunning test_should_pretty_format_even_when_LANG_is_unset ... SUCCESSOverall result: SUCCESSYou might also want to run only specific tests, you may do so with the-p option. This option accepts a pattern as parameter and filters testfunctions against this pattern.
./bash_unit -p fail_fails -p assert tests/test_core.shRunning tests in tests/test_core.shRunning test_assert_equals_fails_when_not_equal ... SUCCESSRunning test_assert_equals_succeed_when_equal ... SUCCESSRunning test_assert_fails ... SUCCESSRunning test_assert_fails_fails ... SUCCESSRunning test_assert_fails_succeeds ... SUCCESSRunning test_assert_matches_fails_when_not_matching ... SUCCESSRunning test_assert_matches_succeed_when_matching ... SUCCESSRunning test_assert_no_diff_fails_when_diff ... SUCCESSRunning test_assert_no_diff_succeeds_when_no_diff ... SUCCESSRunning test_assert_not_equals_fails_when_equal ... SUCCESSRunning test_assert_not_equals_succeeds_when_not_equal ... SUCCESSRunning test_assert_not_matches_fails_when_matching ... SUCCESSRunning test_assert_not_matches_succeed_when_not_matching ... SUCCESSRunning test_assert_shows_stderr_on_failure ... SUCCESSRunning test_assert_shows_stdout_on_failure ... SUCCESSRunning test_assert_status_code_fails ... SUCCESSRunning test_assert_status_code_succeeds ... SUCCESSRunning test_assert_succeeds ... SUCCESSRunning test_assert_within_delta_fails ... SUCCESSRunning test_assert_within_delta_succeeds ... SUCCESSRunning test_fail_fails ... SUCCESSOverall result: SUCCESSYou can combine the-p option with-s to skip some of the tests. This option accepts a patternas parameter and mark as skipped any test function which matches this pattern.
./bash_unit -p fail_fails -p assert -s no -s status tests/test_core.shRunning tests in tests/test_core.shRunning test_assert_equals_fails_when_not_equal ... SKIPPEDRunning test_assert_matches_fails_when_not_matching ... SKIPPEDRunning test_assert_no_diff_fails_when_diff ... SKIPPEDRunning test_assert_no_diff_succeeds_when_no_diff ... SKIPPEDRunning test_assert_not_equals_fails_when_equal ... SKIPPEDRunning test_assert_not_equals_succeeds_when_not_equal ... SKIPPEDRunning test_assert_not_matches_fails_when_matching ... SKIPPEDRunning test_assert_not_matches_succeed_when_not_matching ... SKIPPEDRunning test_assert_status_code_fails ... SKIPPEDRunning test_assert_status_code_succeeds ... SKIPPEDRunning test_assert_equals_succeed_when_equal ... SUCCESSRunning test_assert_fails ... SUCCESSRunning test_assert_fails_fails ... SUCCESSRunning test_assert_fails_succeeds ... SUCCESSRunning test_assert_matches_succeed_when_matching ... SUCCESSRunning test_assert_shows_stderr_on_failure ... SUCCESSRunning test_assert_shows_stdout_on_failure ... SUCCESSRunning test_assert_succeeds ... SUCCESSRunning test_assert_within_delta_fails ... SUCCESSRunning test_assert_within_delta_succeeds ... SUCCESSRunning test_fail_fails ... SUCCESSOverall result: SUCCESSbash_unit supports theTest Anything Protocol so you can ask for a tap formattedoutput with the-f option.
./bash_unit -f tap tests/test_core.sh# Running tests in tests/test_core.shok - test_assert_equals_fails_when_not_equalok - test_assert_equals_succeed_when_equalok - test_assert_failsok - test_assert_fails_failsok - test_assert_fails_succeedsok - test_assert_matches_fails_when_not_matchingok - test_assert_matches_succeed_when_matchingok - test_assert_no_diff_fails_when_diffok - test_assert_no_diff_succeeds_when_no_diffok - test_assert_not_equals_fails_when_equalok - test_assert_not_equals_succeeds_when_not_equalok - test_assert_not_matches_fails_when_matchingok - test_assert_not_matches_succeed_when_not_matchingok - test_assert_shows_stderr_on_failureok - test_assert_shows_stdout_on_failureok - test_assert_status_code_failsok - test_assert_status_code_succeedsok - test_assert_succeedsok - test_assert_within_delta_failsok - test_assert_within_delta_succeedsok - test_fail_failsok - test_fail_prints_failure_messageok - test_fail_prints_where_is_errorok - test_fake_actually_fakes_the_commandok - test_fake_can_fake_inlineok - test_fake_echo_stdin_when_no_paramsok - test_fake_exports_faked_in_subshellsok - test_fake_transmits_params_to_fake_codeok - test_fake_transmits_params_to_fake_code_as_arrayok - test_should_pretty_format_even_when_LANG_is_unset1..30Write your test functions in a file. The name of a test function has to start withtest. Only functions starting withtest will be tested.
Use thebash_unit assertion functions in your test functions, see below.
You may write asetup function that will be executed before each test is run.
You may write ateardown function that will be executed after each test is run.
You may write asetup_suite function that will be executed only once before all the tests of your test file.
You may write ateardown_suite function that will be executed only once after all the tests of your test file.
If you write code outside of any bash function, this code will be executed once at test file loading time sinceyour file is a bash script andbash_unit sources it before running your tests. It is suggested to write asetup_suite function and avoid any code outside a bash function. you must not use any bash_unit assertionin setup_suite or use exit in setup_suite for teardown_suite to be run.Seeissue 43 for more details.
If you want to keep an eye on a test not yet implemented, prefix the name of the function bytodo instead of test.Test to do are not executed and do not impact the global status of your test suite but are displayed inbash_unit output.
bash_unit changes the current working directory to the one of the running test file. If you need to access files from your test code, for instance the script under test, use path relative to the test file.
You may need to change the behavior of some commands to create conditions for your code under test to behave as expected. Thefake function may help you to do that, see below.
bash_unit supports several shell oriented assertion functions.
fail [message]
Fails the test and displays an optional message.
test_can_fail() { fail "this test failed on purpose"}Running test_can_fail ... FAILUREthis test failed on purposedoc:2:test_can_fail()assert <assertion> [message]
Evaluatesassertion and fails ifassertion fails.
assertion fails if its evaluation returns a status code different from 0.
In case of failure, the standard output and error of the evaluatedassertion is displayed. The optional message is also displayed.
test_assert_fails() { assert false "this test failed, obviously"}test_assert_succeed() { assert true}Running test_assert_fails ... FAILUREthis test failed, obviouslydoc:2:test_assert_fails()Running test_assert_succeed ... SUCCESSBut you probably want to assert less obvious facts.
code() { touch /tmp/the_file}test_code_creates_the_file() { code assert "test -e /tmp/the_file"}test_code_makes_the_file_executable() { code assert "test -x /tmp/the_file" "/tmp/the_file should be executable"}Running test_code_creates_the_file ... SUCCESSRunning test_code_makes_the_file_executable ... FAILURE/tmp/the_file should be executabledoc:14:test_code_makes_the_file_executable()It may also be fun to use assert to check for the expected content of a file.
code() { echo 'not so cool' > /tmp/the_file}test_code_write_appropriate_content_in_the_file() { code assert "diff <(echo 'this is cool') /tmp/the_file"}Running test_code_write_appropriate_content_in_the_file ... FAILUREout> 1c1out> < this is coolout> ---out> > not so cooldoc:8:test_code_write_appropriate_content_in_the_file()assert_fail <assertion> [message]
Asserts thatassertion fails. This is the opposite ofassert.
assertion fails if its evaluation returns a status code different from 0.
If the evaluated expression does not fail, thenassert_fail will fail and display the standard output and error of the evaluatedassertion. The optional message is also displayed.
code() { echo 'not so cool' > /tmp/the_file}test_code_does_not_write_cool_in_the_file() { code assert_fails "grep cool /tmp/the_file" "should not write 'cool' in /tmp/the_file"}test_code_does_not_write_this_in_the_file() { code assert_fails "grep this /tmp/the_file" "should not write 'this' in /tmp/the_file"}Running test_code_does_not_write_cool_in_the_file ... FAILUREshould not write 'cool' in /tmp/the_fileout> not so cooldoc:8:test_code_does_not_write_cool_in_the_file()Running test_code_does_not_write_this_in_the_file ... SUCCESSassert_status_code <expected_status_code> <assertion> [message]
Checks for a precise status code of the evaluation ofassertion.
It may be useful if you want to distinguish between several error conditions in your code.
In case of failure, the standard output and error of the evaluatedassertion is displayed. The optional message is also displayed.
code() { exit 23}test_code_should_fail_with_code_25() { assert_status_code 25 code}Running test_code_should_fail_with_code_25 ... FAILURE expected status code 25 but was 23doc:6:test_code_should_fail_with_code_25()assert_equals <expected> <actual> [message]
Asserts for equality of the two stringsexpected andactual.
test_obvious_inequality_with_assert_equals(){ assert_equals "a string" "another string" "a string should be another string"}test_obvious_equality_with_assert_equals(){ assert_equals a a}Running test_obvious_equality_with_assert_equals ... SUCCESSRunning test_obvious_inequality_with_assert_equals ... FAILUREa string should be another string expected [a string] but was [another string]doc:2:test_obvious_inequality_with_assert_equals()assert_not_equals <unexpected> <actual> [message]
Asserts for inequality of the two stringsunexpected andactual.
test_obvious_equality_with_assert_not_equals(){ assert_not_equals "a string" "a string" "a string should be different from another string"}test_obvious_inequality_with_assert_not_equals(){ assert_not_equals a b}Running test_obvious_equality_with_assert_not_equals ... FAILUREa string should be different from another string expected different value than [a string] but was the samedoc:2:test_obvious_equality_with_assert_not_equals()Running test_obvious_inequality_with_assert_not_equals ... SUCCESSassert_matches <expected-regex> <actual> [message]
Asserts that the stringactual matches the regex patternexpected-regex.
test_obvious_notmatching_with_assert_matches(){ assert_matches "a str.*" "another string" "'another string' should not match 'a str.*'"}test_obvious_matching_with_assert_matches(){ assert_matches "a[nN].t{0,1}.*r str.*" "another string"}Running test_obvious_matching_with_assert_matches ... SUCCESSRunning test_obvious_notmatching_with_assert_matches ... FAILURE'another string' should not match 'a str.*' expected regex [a str.*] to match [another string]doc:2:test_obvious_notmatching_with_assert_matches()assert_not_matches <unexpected-regex> <actual> [message]
Asserts that the stringactual does not match the regex patternunexpected-regex.
test_obvious_matching_with_assert_not_matches(){ assert_not_matches "a str.*" "a string" "'a string' should not match 'a str.*'"}test_obvious_notmatching_with_assert_not_matches(){ assert_not_matches "a str.*" "another string"}Running test_obvious_matching_with_assert_not_matches ... FAILURE'a string' should not match 'a str.*' expected regex [a str.*] should not match but matched [a string]doc:2:test_obvious_matching_with_assert_not_matches()Running test_obvious_notmatching_with_assert_not_matches ... SUCCESSassert_within_delta <expected num> <actual num> <max delta> [message]
Asserts that the expected num matches the actual num up to a given max delta.This function only support integers.Given an expectation of 5 and a delta of 2 this would match 3, 4, 5, 6, and 7:
test_matches_within_delta(){ assert_within_delta 5 3 2 assert_within_delta 5 4 2 assert_within_delta 5 5 2 assert_within_delta 5 6 2 assert_within_delta 5 7 2}test_does_not_match_within_delta(){ assert_within_delta 5 2 2}Running test_does_not_match_within_delta ... FAILURE expected value [5] to match [2] with a maximum delta of [2]doc:9:test_does_not_match_within_delta()Running test_matches_within_delta ... SUCCESSassert_no_diff <expected> <actual> [message]
Asserts that the content of the fileactual does not have any differences to the oneexpected.
test_obvious_notmatching_with_assert_no_diff(){ assert_no_diff <(echo foo) <(echo bar)}test_obvious_matching_with_assert_assert_no_diff(){ assert_no_diff bash_unit bash_unit}Running test_obvious_matching_with_assert_assert_no_diff ... SUCCESSRunning test_obvious_notmatching_with_assert_no_diff ... FAILURE expected 'doc' to be identical to 'doc' but was differentout> 1c1out> < fooout> ---out> > bardoc:2:test_obvious_notmatching_with_assert_no_diff()skip_if <condition> <pattern>
Ifcondition is true, will skip all the tests in the current file which match the givenpattern.
This can be useful when one has tests that are dependent on system environment, for instance:
skip_if "uname | grep Darwin" linuxskip_if "uname | grep Linux" darwintest_linux_proc_exists() { assert "ls /proc/" "there should exist /proc on Linux"}test_darwin_proc_does_not_exist() { assert_fail "ls /proc/" "there should not exist /proc on Darwin"}will output, on a Linux system:
Running test_darwin_proc_does_not_exist ... SKIPPEDRunning test_linux_proc_exists ... SUCCESSfake <command> [replacement code]
Fakescommand and replaces it withreplacement code (if code is specified) for the rest of the execution of your test. If no replacement code is specified, then it replaces command by one that echoes stdin of fake. This may be useful if you need to simulate an environment for you code under test.
For instance:
fake ps echo hello worldpswill output:
hello worldWe can do the same usingstdin of fake:
fake ps << EOFhello worldEOFpshello worldIt has been asked whether usingfake results in creating actual fakes or stubs or mocks? or may be spies? or may be they are dummies?The first answer to this question is: it depends. The second is: read thisgreat and detailed literature on this subject.
Here is an example, parameterizing fake with itsstdin to test that code fails when some process does not run and succeeds otherwise:
code() { ps a | grep apache}test_code_succeeds_if_apache_runs() { fake ps <<EOF PID TTY TIME CMD13525 pts/7 00:00:01 bash24162 pts/7 00:00:00 ps 8387 ? 0:00 /usr/sbin/apache2 -k startEOF assert code "code should succeed when apache is running"}test_code_fails_if_apache_does_not_run() { fake ps <<EOF PID TTY TIME CMD13525 pts/7 00:00:01 bash24162 pts/7 00:00:00 psEOF assert_fails code "code should fail when apache is not running"}Running test_code_fails_if_apache_does_not_run ... SUCCESSRunning test_code_succeeds_if_apache_runs ... SUCCESSIn a previous example, we fakedps by specifying code inline:
fake ps echo hello worldpshello worldIf you need to write more complex code to fake your command, you may abstract this code in a function:
_ps() { echo hello world}fake ps _pspshello worldBe careful however that your _ps function is not exported to sub-processes. It means that, depending on how your code under test works, _ps may not be defined in the context where ps will be called. For instance:
_ps() { echo hello world}fake ps _psbash -c psenvironment: line 1: _ps: command not foundIt depends on your code under test but it is safer to just export functions needed by your fake so that they are available in sub-processes:
_ps() { echo hello world}export -f _psfake ps _psbash -c pshello worldfake is also limited by the fact that it defines abash function tooverride the actual command. In some context the command can not beoverridden by a function. For instance if your code under test relies onexec to launchps,fake will have no effect.
fake may also imply strange behaviors from bash_unit when you try tofake really basic stuff. bash_unit tries to be as much immune to this aspossible but there are some limits. Especially and as surprising as itmight seem, bash allows creating functions named after builtin commandsand bash_unit won’t resist that kind of situation. So, for instance, donot try to fake:exit;local;trap;eval;export;if;then;else;fi;while;do;done;$;echo;[ (I know, this is not a builtin but don’t).
fake stores parameters given to the fake in the global variableFAKE_PARAMS so that you can use them inside your fake.
It may be useful if you need to adapt the behavior on the given parameters.
It can also help in asserting the values of these parameters … but this may be quite tricky.
For instance, in our previous code that checks apache is running, we have an issue since our code does not useps with the appropriate parameters. So we will try to check that parameters given to ps areax.
To do that, the first naive approach would be:
code() { ps a | grep apache}test_code_gives_ps_appropriate_parameters() { _ps() { cat <<EOF PID TTY TIME CMD13525 pts/7 00:00:01 bash24162 pts/7 00:00:00 ps 8387 ? 0:00 /usr/sbin/apache2 -k startEOF assert_equals ax "${FAKE_PARAMS[@]}" } export -f _ps fake ps _ps code >/dev/null}This test callscode, which callsps, which is actually implemented by_ps. Sincecode does not useax but onlya as parameters, this test should fail. But …
Running test_code_gives_ps_appropriate_parameters ... SUCCESSThe problem here is thatps fail (because of the failedassert_equals assertion). Butps is piped withgrep:
code() { ps a| grep apache}
With bash, the result code of a pipeline equals the result code of the last command of the pipeline. The last command isgrep and since grep succeeds, the failure of_ps is lost and our test succeeds. We have only succeeded in messing with the test output, nothing more.
An alternative may be to activate bashpipefail option but this may introduce unwanted side effects. We can also simply not output anything in_ps so thatgrep fails:
code() { ps a| grep apache}test_code_gives_ps_appropriate_parameters() {_ps() { assert_equals ax"${FAKE_PARAMS[@]}" }export -f _ps fake ps _ps code>/dev/null}
The problem here is that we use a trick to make the code under test fail but thefailure has nothing to do with the actualassert_equals failure. This is reallybad, don’t do that.
Moreover,assert_equals output is captured byps and this just messes with the display of our test results:
Running test_code_gives_ps_appropriate_parameters ...
The only correct alternative is for the fakeps to writeFAKE_PARAMS in a file descriptorso that your test can grab them after code execution and assert their value. For instanceby writing to a file:
code() { ps a | grep apache}test_code_gives_ps_appropriate_parameters() { _ps() { echo ${FAKE_PARAMS[@]} > /tmp/fake_params } export -f _ps fake ps _ps code || true assert_equals ax "$(head -n1 /tmp/fake_params)"}setup() { rm -f /tmp/fake_params}Here our fake writes to/tmp/fake. We delete this file insetup to besure that we do not get inappropriate data from a previous test. We assertthat the first line of/tmp/fake equalsax. Also, note that we knowthatcode will fail and write this to ignore the error:code || true.
Running test_code_gives_ps_appropriate_parameters ... FAILURE expected [ax] but was [a]doc:14:test_code_gives_ps_appropriate_parameters()We can also compact the fake definition:
code() { ps a | grep apache}test_code_gives_ps_appropriate_parameters() { fake ps 'echo ${FAKE_PARAMS[@]} >/tmp/fake_params' code || true assert_equals ax "$(head -n1 /tmp/fake_params)"}setup() { rm -f /tmp/fake_params}Running test_code_gives_ps_appropriate_parameters ... FAILURE expected [ax] but was [a]doc:10:test_code_gives_ps_appropriate_parameters()Finally, we can avoid the/tmp/fake_params temporary file by usingcoproc:
code() { ps a | grep apache}test_get_data_from_fake() { #Fasten you seat belt ... coproc cat exec {test_channel}>&${COPROC[1]} fake ps 'echo ${FAKE_PARAMS[@]} >&$test_channel' code || true assert_equals ax "$(head -n1 <&${COPROC[0]})"}Running test_get_data_from_fake ... FAILURE expected [ax] but was [a]doc:13:test_get_data_from_fake()About
bash unit testing enterprise edition framework for professionals
Topics
Resources
License
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Sponsor this project
Uh oh!
There was an error while loading.Please reload this page.
Packages0
Uh oh!
There was an error while loading.Please reload this page.