Repository files navigation

An easy to use and lightweight unit test framework forC/C++ onLinux exported as a single header fileeztest.h

The API is made to mostly matchGoogleTest. For someprojects it can be a drop-in replacements, for others it won't work.

The key selling point, and primary motivating factor for writingEZTest, is that it will work out of the box with-fsanitizer=memory,as opposed toGoogleTestwhich requires re-building from source along with an-fsanitizer=memoryenabledlibc++ to work.

• 1. Overview and Motivation
• 2. Installation
  • 2.1. Installing Using CMake
  • 2.2. Installing As Submodule
  • 2.3. Advanced Installation
• 3. Usage
  • 3.1. Namespacing
  • 3.2. Running Tests
  • 3.3. Test Setup
  • 3.4. Asserts and Expects
  • 3.5. Printing On Failure
  • 3.6. Disabling Tests
  • 3.7. Difference From GoogleTest
• 4. Advanced Usage and Toggles
  • 4.1. Behavioral Toggles
  • 4.2. Compilation Toggles
• 5. System Requirements
  • 5.1. Operating System
  • 5.2. Compiler
  • 5.3. Language
  • 5.4. Architecture
• 6. Developer Info
  • 6.1. EZTest Source
  • 6.2. EZTest Header Generation
  • 6.3. Internal Testing of EZTest

The general motivating factor forEZTest is to be an alternative toGoogleTest with a fewimportant distinguishing characteristics.

1. EZTest tests do not link against the C++ runtime library(libc++.so/libstdc++.so). This makes compiling the tests with-fsanitizer=memorytrivial.
2. EZTest runs each test as its own process (usingfork). Thisallows for signal handling of buggy tests with clear errormessages. Further it isolates tests from one another preventingpotential corruption carrying over from one test to another.
3. EZTest is packages as just a single header file and requires noextra link/compilation steps to get working.

The default installation location is/usr/local.

Keep in mind, the only file necessary to get started iseztest.h,so feel free to just copy and include it as you see fit.

Clone the repository and run these commands in the cloned folder:

mkdir build&&cd buildcmake ..# Use -DCMAKE_INSTALL_PREFIX=<your_path> to control the destinationcmake --build. --target install

Then to use the installed library, add the following to yourcmake project:

find_package(eztest REQUIRED)target_link_libraries(your_test_target eztest::eztest)

Clone the repository and add the following to yourcmake project

add_subdirectory(path/to/your/cloned/eztest)target_link_libraries(your_test_target eztest)

There are several toggles for modify how theeztest.his generated/installed.

These include

option(  EZTEST_ULP_PRECISION"Int: Set float/double compare ULP bound"OFF)option(  EZTEST_FLOAT_ULP_PRECISION"Int: Set float compare ULP bound"OFF)option(  EZTEST_DOUBLE_ULP_PRECISION"Int: Set double compare ULP bound"OFF)option(  EZTEST_DISABLE_WARNINGS"Bool: Set/unset to configure whether warnings are supressed with pragmas"OFF)option(  EZTEST_DISABLE_LINTS"Bool: Set/unset to configure whether lints are supressed with comments"OFF)option(  EZTEST_STRICT_NAMESPACE"Bool: Set/unset to configure whether generic TEST/ASSERT/EXPECT macros are defined"OFF)

and can be set during the installation setup. All of these other thanEZTEST_DISABLE_LINTS have corresponding macros (see4. AdvancedUsage and Toggles). TheEZTEST_DISABLE_LINTS option will determine whether/* NOLINT* */directives are transferred from the source code to the header duringgeneration. If you want to ensure certain code characteristics usingclang-tidy including in theEZTest header, set-DEZTEST_DISABLE_LINTS=ON during installation.

The API is similiar to that ofGoogleTest but there aresome key distinction.

All symbols and macros defined ineztest.hare prefixedeztest orEZTEST. Furthermore if compiling withC++all symbols are inside theeztest:: namespace. Assuming your codehas no symbols/macros begining witheztest orEZTEST and/or nosymbols in theeztest:: namespace, there should be no symbolconflicts when usingEZTest.

1. Includeeztest.h in your test file
2. Build your test executable
3. Run the resulting executable.

For example take the followingtest.cc

/* eztest.h contains `main`.  */#include"eztest/eztest.h"TEST(foo,bar) {ASSERT_EQ(0,0);}

To run the test(s) we would do the following:

clang++ test.cc -O3 -otest./test

Tests are created with the following macros:

• TEST(suite, name)
• TEST_TIMED(suite, name, timeout_in_milliseconds)

The usage is identical toGoogleTest i.e:

TEST(my_suite,my_test) {/* Test code goes here... */}

There are assertions / expect statements a variety of differentchecks. The difference between anASSERT check andEXPECT check,is that if anASSERT check fails, the test will endimmediately. Alternatively if anEXPECT check fails, the test willcontinue running but will fail on termination.

• {ASSERT|EXPECT}_TRUE(arg:bool)
  • Checks thearg istrue
• {ASSERT|EXPECT}_FALSE(arg:bool)
  • Checks thearg isfalse
• {ASSERT|EXPECT}_EQ(lhs:anyT, rhs:anyT)
  • Checks thatlhs == rhs
• {ASSERT|EXPECT}_NE(lhs:anyT, rhs:anyT)
  • Checks thatlhs != rhs
• {ASSERT|EXPECT}_LE(lhs:anyT, rhs:anyT)
  • Checks thatlhs <= rhs
• {ASSERT|EXPECT}_LT(lhs:anyT, rhs:anyT)
  • Checks thatlhs < rhs
• {ASSERT|EXPECT}_GE(lhs:anyT, rhs:anyT)
  • Checks thatlhs >= rhs
• {ASSERT|EXPECT}_GT(lhs:anyT, rhs:anyT)
  • Checks thatlhs > rhs
• {ASSERT|EXPECT}_STREQ(lhs:str, rhs:str)
  • Checks thatstrcmp(lhs, rhs) == 0
• {ASSERT|EXPECT}_STRNE(lhs:str, rhs:str)
  • Checks thatstrcmp(lhs, rhs) != 0
• {ASSERT|EXPECT}_STRCASEEQ(lhs:str, rhs:str)
  • Checks thatstrcasecmp(lhs, rhs) == 0
• {ASSERT|EXPECT}_STRCASENE(lhs:str, rhs:str)
  • Checks thatstrcasecmp(lhs, rhs) != 0
• {ASSERT|EXPECT}_FLOAT_EQ(lhs:float, rhs:float)
  • Checks thatULP_difference(lhs, rhs) < Threshold
    • See more onULP.
• {ASSERT|EXPECT}_DOUBLE_EQ(lhs:double, rhs:double)
  • Checks thatULP_difference(lhs, rhs) < Threshold
• {ASSERT|EXPECT}_NEAR(lhs:fp, rhs:fp, bound:fp)
  • Checks thatabs(lhs - rhs) <= bound

In the above:

• bool is an type that supports the! operator.
• anyT is any type that the specified operator is valid for.
• str is one of the following:
  • char *
  • std::string (if usingC++)
  • std::string_view (if usingC++17 or newer)
• fp is one of the following:
  • float
  • double

Failure messages that print the variables exist in the followingcases:

• You are usingC++
• You are usingC11 or newer with a compiler that supports the__typeof__extension.

Otherwise, the failure message will only indicate the line number /variables names that failed.

In additional to default failure messages, you can also optionallyincludeprintf asoptional additional arguments to anyASSERT/EXPECT macro that willbe printed only on failure. For example:

/* int a, b, c; */intd=a+b;ASSERT_EQ(c,d,"Something something %d + %d\n",a,b);

Tests can be disabled from running (but still built) by prefixing thetest name withDISABLED_.

This behaves exactly the same asGoogleTest.

There are several key distinctions between theEZTest API and thatofGoogleTest.

In no particularly the notable differences are:

*Support/fix is planned.

1. *The following macros are unimplemented inEZTest:

   • TEST_F
   • TEST_P
   • SCOPED_TRACE
   • {ASSERT|EXPECT}_THROW
   • {ASSERT|EXPECT}_ANY_THROW
   • {ASSERT|EXPECT}_NO_THROW
   • {ASSERT|EXPECT}_NO_FATAL_FAILURE
   • {ASSERT|EXPECT}_PRED1
   • {ASSERT|EXPECT}_PRED2
   • {ASSERT|EXPECT}_PRED3
   • {ASSERT|EXPECT}_PRED4
   • {ASSERT|EXPECT}_PRED5

2. There is no testFixture toinherit from inEZTest.

3. EZTest usesC-style printing as opposed toC++-style operator<<printing.

4. *There are noDeathTests inEZTest.

5. *There is an environment variable / commandline support inEZTest. This is relevant to

   • Test filtering.
   • Test repeating
   • Test shuffling
   • Test listing
   • etc...

6. *There are no toggles for modify test printouts inEZTest.

7. *There is no support for generating an XML/JSON files for the test results inEZTest.

8. *There are slight differences in the printout format.

9. *No global variables for changing behavior

   • Instead there are some macros.

There are some macros that can be defined which will change thebehavior/compilation ofeztest.h.

For the most part, the defaults should be fine.

The following macros can be defined to change behavior.

1. EZTEST_ULP_PRECISION
   • default:4
   • This will change theULPfor both{ASSERT|EXPECT}_{FLOAT|DOUBLE}_EQ
2. EZTEST_FLOAT_ULP_PRECISION
   • default:EZTEST_ULP_PRECISION
   • This will change theULPfor both{ASSERT|EXPECT}_FLOAT_EQ
3. EZTEST_DOUBLE_ULP_PRECISION
   • default:EZTEST_ULP_PRECISION
   • This will change theULPfor both{ASSERT|EXPECT}_DOUBLE_EQ
4. EZTEST_C_PRINT_ARGS
   • default:1
   • If set to0 and using theC language,eztest will stoptrying to print arguments on failure.
5. EZTEST_VERBOSITY
   • default:0
   • Increase value to include more internal printouts (mostly justnon-fatal warnings).

The following macros can be defined to change compilation.

1. EZTEST_DISABLE_WARNINGS

   • default:1
   • If set,eztest will enablePragmas todisable known, inevitable, warnings. These warnings mostly onlyshow up if compiling withClang's -Weverythingor an excessive amount ofGCC warnings. If compiling with-Wall -Wextra -Wpedantic the only warning that may show up is-Wunused-function depending on the set ofASSERT/EXPECTchecks used. The total set of disabled warnings are:
     • -Waggregate-returns
     • -Wcxx98-compat-pedantic
     • -Wcxx98-compat
     • -Wdouble-promotion
     • -Wfloat-equal
     • -Wformat-nonliteral
     • -Wglobal-constructors
     • -Wpadded
     • -Wunsafe-buffer-usage
     • -Wunsafe-buffer-usage-in-libc-call
     • -Wunused-function
     • -Wunused-member-function
     • -Wunused-result
     • -Wunused-template
   • Note thePragmas are used minimally and will never apply toyour own code.

2. EZTEST_STRICT_NAMESPACE

   • default:0
   • If set, then general{ASSERT|EXPECT}_* andTEST* macros willbe prefixed withEZTEST_

There are some system requirements. Some of these may not behard-requirements, but at the very least they are untested.

Linux is the only tested OS at the moment. It's probably that anyunix based system would work.

GCC/Clang are the only tested compilers are the moment.

1. IfC
   • C99 or newer
   • Support for__attribute__((constructor))
2. IfC++
   • C++11 or newer

If compilingwithoutposix specification-x86-64-x86-32-arm-aarch64-riscv64-riscv32

Otherwise any target should work.

The source code is located insrc/eztest.

The source code and includableeztest.hfiles seperate. The actualeztest.hheader is autogenerated. This is a convenience to make developmentsimpler.

To generate theeztest.hheader file, we use thescripts/freeze.pyscript.

Generally there is no need to invoke the script directly. It willeither be invoked bycmake as needed for tests/installation, or bythe wrapper scriptquick-regen.py.

Tests are located in thetestsdirectory.

To build the tests you can run:

mkdir build&&cd buildcmake .. -DEZTEST_BUILD_TESTS=ONmake check-all# Run "unit tests"make check-external# Run "integration tests"make run-static-analysis# Run clang-tidy

The generalcmake compiler/flags/language arguments will alsoapply. As well there are toolchain files are testing crosscompilation.

To see some examples of the internal tests usage see.github/workflows/ci.yaml`.