- Notifications
You must be signed in to change notification settings - Fork11
Dead simple testing framework for Bash with coverage reporting
License
Checksum/critic.sh
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Dead simple testing framework for Bash with coverage.
I was looking for a Bash testing framework with a familiar API and with coverage reporting. Although there are excellent frameworks likebats-core,shunit2 andbashunit, I wasn't too comfortable with their API (not their fault). Also, I wanted some indication of coverage, so that it can be improved over time.
critic.sh
exposes high level functions for testing consistent with other frameworks and a set of built in assertions. One of my most important goals was to be able to pass in any shell expression to the_test
and_assert
methods, so that one is not limited to the built-ins.
In addition, it can generate a lcov report. It tracks line and function coverage, but not branches. It works by running the tests with extended debugging, redirecting the trace output to a log file, and then parsing it to determine which functions/lines have been executed. It is currently a work in progress.
Due to use of certain bashisms, Bash v4.1+ is required. This may change in the future.
A tiny docker image is provided for convenience.
There are a few ways to usecritic.sh
:
- Use the docker image
docker run --rm -v$(pwd):/work checksum/critic.sh'/work/src/*.sh''/work/lib/*.sh'
You can pass aCRITIC_SETUP
environment variable to run setup scripts before the tests are run. The docker image is based on alpine linux, so useapk
to install packages:
docker run --rm -e CRITIC_SETUP='apk add --no-cache jq' -v$(pwd):/work checksum/critic.sh'/work/src/*.sh''/work/lib/*.sh'
- Add this repository as a git submodule in your project
git submodule add https://github.com/Checksum/critic.sh criticcritic/critic.sh test.sh
- Copy
critic.sh
file into your project (not recommended)
Seeexamples/test.sh
for detailed usage. To run the tests:bash examples/test.sh
# test-foobar.sh# Include your source filessource foobar.sh# Include the frameworksource critic.sh# Write tests_describe foo _test"output should equal foo" _assert _output_equals"foo" _test"return code should be 0" _assert _return_true"Optional assertion message"
critic.sh test-foobar.sh
The layout of a test is consistent with other frameworks. You_describe
a test suite,_test
a function or expression, and_assert
the output with a function or expression. The output, return code and arguments passed to the test are available as variables for all custom assertions.
Function | Description | Arguments |
---|---|---|
_describe | Run test suite | 1. Suite/Function name (*) |
_describe_skip | Skip this test suite | 1. Suite/Function name (*) |
_test | Run a test | 1. Test name (*) |
2. Test function/expression | ||
3. Arguments to forward to the test function | ||
_test_skip | Skip this test | 1. Test name (*) |
2. Test function/expression | ||
3. Arguments to forward to the test function | ||
_assert | Run an assertion | 1. Assertion function/expression (*) |
2. Arguments to forward to the assertion function | ||
_teardown | Teardown function run after all tests have ben run |
Function | Description | Arguments |
---|---|---|
_return_true | Return code == 0 | 1. Optional message |
_return_false | Return code != 0 | 1. Optional message |
_return_equals | Return code == num | 1. Return code (*) |
2. Optional message | ||
_output_contains | Output contains value | 1. Value (*) |
2. Optional message | ||
_output_equals | Output equals value | 1. Value (*) |
2. Optional message | ||
_not | Negate an assertion | 1. Assertion (*) |
2. Value (*) | ||
3. Optional message | ||
_nth_arg_equals | Nth arg equals value | 1. Argument index (>=0) |
2. Value | ||
3. Optional message |
After every_test
is run, the following variables are set. These are useful for custom assertions:
Variable | Description |
---|---|
_output | Output of the function/expression |
_return | Return code |
_args | Arguments passed to the function/expression |
Environment variable | Description | Default |
---|---|---|
CRITIC_COVERAGE_DISABLE | Disable coverage | false |
CRITIC_COVERAGE_MIN_PERCENT | Minimum coverage percent per source file | 0 |
CRITIC_COVERAGE_REPORT_CLI | Print coverage report to CLI | true |
CRITIC_COVERAGE_REPORT_LCOV | Save lcov report | true |
CRITIC_COVERAGE_REPORT_HTML | Generate HTML lcov report (requireslcov ) | false |
CRITIC_DEBUG | Prints more verbose messages | false |
# critic ignorefoo() {echo"This function will skipped when calculating coverage %"}# critic /ignore
About
Dead simple testing framework for Bash with coverage reporting