The Groovy programming language comes with great support for writing tests. In addition to the languagefeatures and test integration with state-of-the-art testing libraries and frameworks, the Groovy ecosystem has borna rich set of testing libraries and frameworks.
This chapter will start with language specific testing features and continue with a closer look at JUnitintegration, Spock for specifications, and Geb for functional tests. Finally, we’ll do an overview of other testinglibraries known to be working with Groovy.
Besides integrated support for JUnit, the Groovy programming language comes with features that have provento be very valuable for test-driven development. This section gives insight on them.
Writing tests means formulating assumptions by using assertions. In Java this can be done by using theassertkeyword that has been added in J2SE 1.4. In Java,assert statements can be enabled via the JVM parameters-ea(or-enableassertions) and-da (or-disableassertions). Assertion statements in Java are disabled by default.
Groovy comes with a ratherpowerful variant ofassert also known aspower assertion statement. Groovy’s powerassert differs from the Java version in its output given the boolean expression validates tofalse:
def x = 1assert x == 2// Output:(1)//// Assertion failed:// assert x == 2// | |// 1 false| 1 | This section shows the std-err output |
Thejava.lang.AssertionError that is thrown whenever the assertion can not be validated successfully, containsan extended version of the original exception message. The power assertion output shows evaluation results fromthe outer to the inner expression.
The power assertion statements true power unleashes in complex Boolean statements, or statements withcollections or othertoString-enabled classes:
def x = [1,2,3,4,5]assert (x << 6) == [6,7,8,9,10]// Output://// Assertion failed:// assert (x << 6) == [6,7,8,9,10]// | | |// | | false// | [1, 2, 3, 4, 5, 6]// [1, 2, 3, 4, 5, 6]Another important difference from Java is that in Groovy assertions areenabled by default. It has been a language designdecision to remove the possibility to deactivate assertions. Or, as Bertrand Meyer stated,it makes no sense to takeoff your swim ring if you put your feet into real water.
One thing to be aware of are methods with side effects inside Boolean expressions in power assertion statements. Asthe internal error message construction mechanism does only store references to instances under target, it happens that the error message text is invalid at rendering time in case of side-effecting methods involved:
assert [[1,2,3,3,3,3,4]].first().unique() == [1,2,3]// Output://// Assertion failed:// assert [[1,2,3,3,3,3,4]].first().unique() == [1,2,3]// | | |// | | false// | [1, 2, 3, 4]// [1, 2, 3, 4](1)| 1 | The error message shows the actual state of the collection, not the state before theunique method was applied |
If you choose to provide a custom assertion error message this can be done by using the Java syntaxassertexpression1 : expression2 whereexpression1 is the Boolean expression andexpression2 is the custom error message. Be aware though that this will disable the power assert and will fully fall back to custom error messages on assertion errors. |
Groovy has excellent built-in support for a range of mocking and stubbing alternatives. When using Java, dynamic mockingframeworks are very popular. A key reason for this is that it is hard work creating custom hand-crafted mocks using Java.Such frameworks can be used easily with Groovy if you choose but creating custom mocks is much easier in Groovy. Youcan often get away with simple maps or closures to build your custom mocks.
The following sections show ways to create mocks and stubs with Groovy language features only.
By using maps or expandos, we can incorporate desired behaviour of a collaborator very easily as shown here:
class TranslationService { String convert(String key) { return "test" }}def service = [convert: { String key -> 'some text' }] as TranslationServiceassert 'some text' == service.convert('key.text')Theas operator can be used to coerce a map to a particular class. The given map keys are interpreted asmethod names and the values, beinggroovy.lang.Closure blocks, are interpreted as method code blocks.
Be aware that map coercion can get into the way if you deal with customjava.util.Map descendant classes in combinationwith theas operator. The map coercion mechanism is targeted directly at certain collection classes, it doesn’t takecustom classes into account. |
The 'as' operator can be used with closures in a neat way which is great for developer testing in simple scenarios.We haven’t found this technique to be so powerful that we want to do away with dynamic mocking, but it can be veryuseful in simple cases none-the-less.
Classes or interfaces holding a single method, including SAM (single abstract method) classes, can be used to coercea closure block to be an object of the given type. Be aware that for doing this, Groovy internally create a proxy objectdescending for the given class. So the object will not be a direct instance of the given class. This important if, forexample, the generated proxy object’s metaclass is altered afterwards.
Let’s have an example on coercing a closure to be of a specific type:
def service = { String key -> 'some text' } as TranslationServiceassert 'some text' == service.convert('key.text')Groovy supports a feature called implicit SAM coercion. This means that theas operator is not necessary in situationswhere the runtime can infer the target SAM type. This type of coercion might be useful in tests to mock entire SAMclasses:
abstract class BaseService { abstract void doSomething()}BaseService service = { -> println 'doing something' }service.doSomething()The Groovy mocking and stubbing classes can be found in thegroovy.mock.interceptor package.
TheMockFor class supports (typically unit) testing of classes in isolation by allowing astrictly ordered expectationof the behavior of collaborators to be defined. A typical test scenario involves a class under test and one or more collaborators. In such a scenario it isoften desirable to just test the business logic of the class under test. One strategy for doing that is to replacethe collaborator instances with simplified mock objects to help isolate out the logic in the test target. MockForallows such mocks to be created using meta-programming. The desired behavior of collaborators is defined as a behaviorspecification. The behavior is enforced and checked automatically.
Let’s assume our target classes looked like this:
class Person { String first, last}class Family { Person father, mother def nameOfMother() { "$mother.first $mother.last" }}WithMockFor, a mock expectation is always sequence dependent and its use automatically ends with a call toverify:
def mock = new MockFor(Person)(1)mock.demand.getFirst{ 'dummy' }mock.demand.getLast{ 'name' }mock.use {(2) def mary = new Person(first:'Mary', last:'Smith') def f = new Family(mother:mary) assert f.nameOfMother() == 'dummy name'}mock.expect.verify()(3)| 1 | a new mock is created by a new instance ofMockFor |
| 2 | aClosure is passed touse which enables the mocking functionality |
| 3 | a call toverify checks whether the sequence and number of method calls is as expected |
TheStubFor class supports (typically unit) testing of classes in isolation by allowing aloosely-ordered expectationof the behavior of collaborators to be defined. A typical test scenario involves a class under test and one or morecollaborators. In such a scenario it is often desirable to just test the business logic of the CUT. One strategy fordoing that is to replace the collaborator instances with simplified stub objects to help isolate out the logicin the target class.StubFor allows such stubs to be created using meta-programming. The desired behavior ofcollaborators is defined as a behavior specification.
In contrast toMockFor the stub expectation checked withverify is sequence independent and its use is optional:
def stub = new StubFor(Person)(1)stub.demand.with {(2) getLast{ 'name' } getFirst{ 'dummy' }}stub.use {(3) def john = new Person(first:'John', last:'Smith') def f = new Family(father:john) assert f.father.first == 'dummy' assert f.father.last == 'name'}stub.expect.verify()(4)| 1 | a new stub is created by a new instance ofStubFor |
| 2 | thewith method is used for delegating all calls inside the closure to theStubFor instance |
| 3 | aClosure is passed touse which enables the stubbing functionality |
| 4 | a call toverify (optional) checks whether the number of method calls is as expected |
MockFor andStubFor can not be used to test statically compiled classes e.g. for Java classes or Groovy classes thatmake use of@CompileStatic. To stub and/or mock these classes you can use Spock or one of the Java mocking libraries.
Groovy includes a specialMetaClass the so-calledExpandoMetaClass (EMC). It allows to dynamically add methods,constructors, properties and static methods using a neat closure syntax.
Everyjava.lang.Class is supplied with a specialmetaClass property that will give a reference to anExpandoMetaClass instance. The expando metaclass is not restricted to custom classes, it can be used forJDK classes like for examplejava.lang.String as well:
String.metaClass.swapCase = {-> def sb = new StringBuffer() delegate.each { sb << (Character.isUpperCase(it as char) ? Character.toLowerCase(it as char) : Character.toUpperCase(it as char)) } sb.toString()}def s = "heLLo, worLD!"assert s.swapCase() == 'HEllO, WORld!'TheExpandoMetaClass is a rather good candidate for mocking functionality as it allows for more advanced stufflike mocking static methods
class Book { String title}Book.metaClass.static.create << { String title -> new Book(title:title) }def b = Book.create("The Stand")assert b.title == 'The Stand'or even constructors
Book.metaClass.constructor << { String title -> new Book(title:title) }def b = new Book("The Stand")assert b.title == 'The Stand'Mocking constructors might seem like a hack that’s better not even to be considered but even there might be validuse cases. An example can be found in Grails where domain class constructors are added at run-time with thehelp ofExpandoMetaClass. This lets the domain object register itself in the Spring application context and allowsfor injection of services or other beans controlled by the dependency-injection container. |
If you want to change themetaClass property on a per test method level you need to remove the changes that weredone to the metaclass, otherwise those changes would be persistent across test method calls. Changes are removed byreplacing the metaclass in theGroovyMetaClassRegistry:
GroovySystem.metaClassRegistry.removeMetaClass(String)Another alternative is to register aMetaClassRegistryChangeEventListener, track the changed classes and removethe changes in the cleanup method of your chosen testing runtime. A good example can be foundin the Grails webdevelopment framework.
Besides using theExpandoMetaClass on a class-level, there is also support for using the metaclass on a per-objectlevel:
def b = new Book(title: "The Stand")b.metaClass.getTitle {-> 'My Title' }assert b.title == 'My Title'In this case the metaclass change is related to the instance only. Depending on the test scenario this might be a betterfit than the global metaclass change.
The following section gives a brief overview on GDK methods that can be leveraged in test case scenarios, for example fortest data generation.
Thecombinations method that is added onjava.lang.Iterable compliant classes can be used to get a list ofcombinations from a list containing two or more sub-lists:
void testCombinations() { def combinations = [[2, 3],[4, 5, 6]].combinations() assert combinations == [[2, 4], [3, 4], [2, 5], [3, 5], [2, 6], [3, 6]]}The method could be used in test case scenarios to generate all possible argument combinations for a specific methodcall.
TheeachCombination method that is added onjava.lang.Iterable can be used to apply a function (or in this case agroovy.lang.Closure) to each if the combinations that has been built by thecombinations method:
eachCombination is a GDK method that is added to all classes conforming to thejava.lang.Iterable interface.It applies a function on each combination of the input lists:
void testEachCombination() { [[2, 3],[4, 5, 6]].eachCombination { println it[0] + it[1] }}The method could be used in the testing context to call methods with each of the generated combinations.
Code coverage is a useful measure of the effectiveness of (unit) tests. A program with high code coverage has alower chance to hold critical bugs than a program with no or low coverage. To get code coverage metrics,the generated byte-code usually needs to be instrumented before the tests are executed. One tool with Groovy support for this task isCobertura.
The following code listing shows an example on how to enable Cobertura test coverage reports in a Gradle build script froma Groovy project:
def pluginVersion = '<plugin version>'def groovyVersion = '<groovy version>'def junitVersion = '<junit version>'buildscript { repositories { mavenCentral() } dependencies { classpath 'com.eriwen:gradle-cobertura-plugin:${pluginVersion}' }}apply plugin: 'groovy'apply plugin: 'cobertura'repositories { mavenCentral()}dependencies { compile "org.codehaus.groovy:groovy-all:${groovyVersion}" testCompile "junit:junit:${junitVersion}"}cobertura { format = 'html' includes = ['**/*.java', '**/*.groovy'] excludes = ['com/thirdparty/**/*.*']}Several output formats can be chosen for Cobertura coverage reports and test code coverage reports can be added tocontinuous integration build tasks.
Groovy simplifies JUnit testing in the following ways:
You use the same overall practices as you would when testing with Java but youcan adopt much of Groovy’s concise syntax in your tests making them succinct. You can even usethe capabilities for writing testing domain specific languages (DSLs) if you feel so inclined.
There are numerous helper classes that simplify many testing activities. The details differin some cases depending on the version of JUnit you are using. We’ll cover those details shortly.
Groovy’s PowerAssert mechanism is wonderful to use in your tests
Groovy deems that tests are so important you should be able to run them as easily as scripts or classes.This is why Groovy includes an automatic test runner when using thegroovy command or the GroovyConsole.This gives you some additional options over and above running your tests
In the following sections we will have a closer look at JUnit 3, 4 and 5 Groovy integration.
Maybe one of the most prominent Groovy classes supporting JUnit 3 tests is theGroovyTestCase class. Beingderived fromjunit.framework.TestCase it offers a bunch of additional methods that make testing in Groovy a breeze.
AlthoughGroovyTestCase inherits fromTestCase doesn’t mean you can’t use JUnit 4 features in your project. In fact,the most recent Groovy versions come with a bundled JUnit 4 and that comes with a backwards compatibleTestCaseimplementation. There have been some discussion on the Groovy mailing-list on whether to useGroovyTestCase or JUnit 4with the result that it is mostly a matter of taste, but withGroovyTestCase you get a bunch of methods for free thatmake certain types of tests easier to write. |
In this section, we will have a look at some of the methods provided byGroovyTestCase. A full list of these can befound in the JavaDoc documentation forgroovy.test.GroovyTestCase ,don’t forget it is inherited fromjunit.framework.TestCase which inherits all theassert* methods.
GroovyTestCase is inherited fromjunit.framework.TestCase therefore it inherits a large number of assertion methodsbeing available to be called in every test method:
class MyTestCase extends GroovyTestCase { void testAssertions() { assertTrue(1 == 1) assertEquals("test", "test") def x = "42" assertNotNull "x must not be null", x assertNull null assertSame x, x }}As can be seen above, in contrast to Java it is possible to leave out the parenthesis in most situations whichleads to even more readability of JUnit assertion method call expressions.
An interesting assertion method that is added byGroovyTestCase isassertScript. It ensures that the given Groovycode string succeeds without any exception:
void testScriptAssertions() { assertScript ''' def x = 1 def y = 2 assert x + y == 3 '''}shouldFail can be used to check whether the given code block fails or not. In case it fails, the assertion does hold,otherwise the assertion fails:
void testInvalidIndexAccess1() { def numbers = [1,2,3,4] shouldFail { numbers.get(4) }}The example above uses the basicshouldFail method interface that takes agroovy.lang.Closure as a single argument. TheClosure instance holds the code that is supposed to be breaking during run-time.
If we wanted to assertshouldFail on a specificjava.lang.Exception type we could have done so by using theshouldFailimplementation that takes theException class as first argument and theClosure as second argument:
void testInvalidIndexAccess2() { def numbers = [1,2,3,4] shouldFail IndexOutOfBoundsException, { numbers.get(4) }}If anything other thanIndexOutOfBoundsException (or a descendant class of it) is thrown, the test case will fail.
A pretty nice feature ofshouldFail hasn’t been visible so far: it returns the exception message. This is reallyuseful if you want to assert on the exception error message:
void testInvalidIndexAccess3() { def numbers = [1,2,3,4] def msg = shouldFail IndexOutOfBoundsException, { numbers.get(4) } assert msg.contains('Index: 4, Size: 4') || msg.contains('Index 4 out-of-bounds for length 4') || msg.contains('Index 4 out of bounds for length 4')}ThenotYetImplemented method has been greatly influenced by HtmlUnit. It allows to write a test method but mark itas not yet implemented. As long as the test method fails and is marked withnotYetImplemented the test goes green:
void testNotYetImplemented1() { if (notYetImplemented()) return(1) assert 1 == 2(2)}| 1 | a call tonotYetImplemented is necessary forGroovyTestCase to get the current method stack. |
| 2 | as long as the test evaluates tofalse the test execution will be successful. |
An alternative to thenotYetImplemented method is the@NotYetImplemented annotation. It allows for annotating amethod as not yet implemented, with the exact same behavior asGroovyTestCase#notYetImplemented but without the needfor thenotYetImplemented method call:
@NotYetImplementedvoid testNotYetImplemented2() { assert 1 == 2}Groovy can be used to write JUnit 4 test cases without any restrictions. Thegroovy.test.GroovyAssert holdsvarious static methods that can be used as replacement for theGroovyTestCase methods in JUnit 4 tests:
import org.junit.Testimport static groovy.test.GroovyAssert.shouldFailclass JUnit4ExampleTests { @Test void indexOutOfBoundsAccess() { def numbers = [1,2,3,4] shouldFail { numbers.get(4) } }}As can be seen in the example above, the static methods found inGroovyAssert are imported at the beginning of the class definition thusshouldFail can be used the same way it can be used in aGroovyTestCase.
groovy.test.GroovyAssert descends fromorg.junit.Assert that means it inherits all JUnit assertion methods. However,with the introduction of the power assertion statement, it turned out to begood practice to rely on assertion statementsinstead of using the JUnit assertion methods with the improved message being the main reason. |
It is worth mentioning thatGroovyAssert.shouldFail is not absolutely identical toGroovyTestCase.shouldFail. WhileGroovyTestCase.shouldFail returns the exception message,GroovyAssert.shouldFail returns the exception itself. Ittakes a few more keystrokes to get the message, but in return you can access other properties and methods of theexception:
@Testvoid shouldFailReturn() { def e = shouldFail { throw new RuntimeException('foo', new RuntimeException('bar')) } assert e instanceof RuntimeException assert e.message == 'foo' assert e.cause.message == 'bar'}Much of the approach and helper classes described under JUnit4 apply when using JUnit5 however JUnit5uses some slightly different class annotations when writing your tests. See theJUnit5 documentationfor more details.
Create your test classes as per normal JUnit5 guidelines as shown in this example:
class MyTest { @Test void streamSum() { assertTrue(Stream.of(1, 2, 3) .mapToInt(i -> i) .sum() > 5, () -> "Sum should be greater than 5") } @RepeatedTest(value=2, name = "{displayName} {currentRepetition}/{totalRepetitions}") void streamSumRepeated() { assert Stream.of(1, 2, 3).mapToInt(i -> i).sum() == 6 } private boolean isPalindrome(s) { s == s.reverse() } @ParameterizedTest(1) @ValueSource(strings = [ "racecar", "radar", "able was I ere I saw elba" ]) void palindromes(String candidate) { assert isPalindrome(candidate) } @TestFactory def dynamicTestCollection() {[ dynamicTest("Add test") { -> assert 1 + 1 == 2 }, dynamicTest("Multiply Test", () -> { assert 2 * 3 == 6 }) ]}}| 1 | This test requires the additionalorg.junit.jupiter:junit-jupiter-params dependency if not already in your project. |
You can run the tests in your IDE or build tool if it supports and is configured for JUnit5.If you run the above test in the GroovyConsole or via thegroovy command, you will see a short text summary of theresult of running the test:
JUnit5 launcher: passed=8, failed=0, skipped=0, time=246ms
More detailed information is available at theFINE logging level. You can configure your logging to display suchinformation or do it programmatically as follows:
@BeforeAllstatic void init() { def logger = Logger.getLogger(LoggingListener.name) logger.level = Level.FINE logger.addHandler(new ConsoleHandler(level: Level.FINE))}Spock is a testing and specification framework for Java and Groovy applications. What makes it stand out from thecrowd is its beautiful and highly expressive specification DSL. In practice, Spock specifications are written asGroovy classes. Although written in Groovy they can be used to test Java classes. Spock can be used for unit,integration or BDD (behavior-driven-development) testing, it doesn’t put itself into a specific category of testingframeworks or libraries.
| Beside these awesome features Spock is a good example on how to leverage advanced Groovy programminglanguage features in third party libraries, for example, by using Groovy AST transformations. |
| This section should not serve as detailed guide on how to use Spock, it should rather give an impression what Spockis about and how it can be leveraged for unit, integration, functional or any other type of testing. |
The next section we will have a first look at the anatomy of a Spock specification. It should give apretty good feeling on what Spock is up to.
Spock lets you write specifications that describe features (properties, aspects) exhibited by a system ofinterest. The "system" can be anything between a single class and an entire application, a more advanced term for it issystem under specification. Thefeature description starts from a specific snapshot of the system and itscollaborators, this snapshot is called thefeature’s fixture.
Spock specification classes are derived fromspock.lang.Specification. A concrete specification class might consistof fields, fixture methods, features methods and helper methods.
Let’s have a look at a simple specification with a single feature method for an imaginaryStack class:
class StackSpec extends Specification { def "adding an element leads to size increase"() {(1) setup: "a new stack instance is created"(2) def stack = new Stack() when:(3) stack.push 42 then:(4) stack.size() == 1 }}| 1 | Feature method, is by convention named with a String literal. |
| 2 | Setup block, here is where any setup work for this feature needs to be done. |
| 3 | When block describes a stimulus, a certain action under target by this feature specification. |
| 4 | Then block any expressions that can be used to validate the result of the code that was triggered by the when block. |
Spock feature specifications are defined as methods inside aspock.lang.Specification class. They describe the featureby using a String literal instead of a method name.
A feature method holds multiple blocks, in our example we usedsetup,when andthen. Thesetup block is specialin that it is optional and allows to configure local variables visible inside the feature method. Thewhen blockdefines the stimulus and is a companion of thethen block which describes the response to the stimulus.
Note that thesetup method in theStackSpec above additionally has a description String. Description Strings are optional and can be added after block labels (likesetup,when,then).
Spock provides much more features like data tables or advanced mocking capabilities. Feel free to consult theSpock GitHub page for more documentation and download information.
Geb is a functional web testing and scraper library that integrates with JUnit and Spock. It is based upon theSelenium web drivers and, like Spock, provides a Groovy DSL to write functional tests for web applications.
Geb has great features that make it a good fit for a functional testing library:
DOM access via a JQuery-like$ function
implements thepage pattern
support for modularization of certain web components (e.g. menu-bars, etc.) withmodules
integration with JavaScript via the JS variable
| This section should not serve as detailed guide on how to use Geb, it should rather give an impression what Gebis about and how it can be leveraged functional testing. |
The next section will give an example on how Geb can be used to write a functional test for a simple web page with a single search field.
Although Geb can be used standalone in a Groovy script, in many scenarios it’s used in combination with other testingframeworks. Geb comes with various base classes that can be used in JUnit 3, 4, TestNG or Spock tests. The base classesare part of additional Geb modules that need to be added as a dependency.
For example, the following@Grab dependencies can be used to run Geb with the Selenium Firefox driver inJUnit4 tests. The module that is needed for JUnit 3/4 support isgeb-junit4:
@Grab('org.gebish:geb-core:0.9.2')@Grab('org.gebish:geb-junit4:0.9.2')@Grab('org.seleniumhq.selenium:selenium-firefox-driver:2.26.0')@Grab('org.seleniumhq.selenium:selenium-support:2.26.0')The central class in Geb is thegeb.Browser class. As its name implies it is used to browse pages and access DOM elements:
import geb.Browserimport org.openqa.selenium.firefox.FirefoxDriverdef browser = new Browser(driver: new FirefoxDriver(), baseUrl: 'http://myhost:8080/myapp')(1)browser.drive { go "/login"(2) $("#username").text = 'John'(3) $("#password").text = 'Doe' $("#loginButton").click() assert title == "My Application - Dashboard"}| 1 | A newBrowser instance is created. In this case it uses the SeleniumFirefoxDriver and sets thebaseUrl. |
| 2 | go is used to navigate to a URL or relative URI |
| 3 | $ together with CSS selectors is used to access theusername andpassword DOM fields. |
TheBrowser class comes with adrive method that delegates all method/property calls to the currentbrowser instance. TheBrowser configuration must not be done inline, it can also be externalized in aGebConfig.groovy configuration file for example. In practice, the usage of theBrowser class is mostly hiddenby Geb test base classes. They delegate all missing properties and method calls to the currentbrowser instancethat exists in the background:
class SearchTests extends geb.junit4.GebTest { @Test void executeSeach() { go 'http://somehost/mayapp/search'(1) $('#searchField').text = 'John Doe'(2) $('#searchButton').click()(3) assert $('.searchResult a').first().text() == 'Mr. John Doe'(4) }}| 1 | Browser#go takes a relative or absolute link and calls the page. |
| 2 | Browser#$ is used to access DOM content. Any CSS selectors supported by the underlying Selenium drivers are allowed |
| 3 | click is used to click a button. |
| 4 | $ is used to get the first link out of thesearchResult block |
The example above shows a simple Geb web test with the JUnit 4 base classgeb.junit4.GebTest. Note that in this casetheBrowser configuration is externalized.GebTest delegates methods likego and$ to the underlyingbrowser instance.
In the previous section we only scratched the surface of the available Geb features. More information on Geb can be foundat theproject homepage.