As a developer, I certainly qualify as a testing freak. I absolutely love automated tests that produce meaningful output. Today, I want to focus on the history and state of the art in automated testing, more specifically: on the assertions.

Disclaimer: we are talking about assertions forTests in this post. Other kinds of assertions (such as pre- and post-conditions and theassert keyword) are beyond the focus of this post.

Some Code to Test

An automated test is not a proper test without at least one assertion (except for smoke tests). For this article, we will test the following Person class:

publicclassPerson{privateStringname;privateDatebirthDate;privatefinalSet<String>hobbies=newHashSet<>();publicPerson(){}publicStringgetName(){returnthis.name;}publicvoidsetName(Stringname){this.name=name;}publicDategetBirthDate(){if(this.birthDate==null){returnnull;}returnnewDate(this.birthDate.getTime());}publicvoidsetBirthDate(Datedate){this.date=newDate(date.getTime());}publicSet<String>getHobbies(){returnCollections.unmodifiableSet(this.hobbies);}publicvoidsetHobbies(Set<String>hobbies){this.hobbies.clear();if(hobbies!=null){this.hobbies.addAll(hobbies);}}}

Disclaimer: I am well aware that testing a bean class and it's accessors is not something one would typically do. However, it serves as a nice, simple example that allows us to focus on the assertions themselves.

The Primordial Assertion

There most basic way to state an assertion in Java (without any frameworks) is the following:

if(someCondition){thrownewAssertionError("Hey I didn't expect that!");}

No matter how fancy your assertion framework is on the surface, in the end it will always boil down to this. There are a couple of ways in which assertion frameworks have improved over this initial assertion method:

• Make it moreconcise. Reduce the amount of code that needs to be written.
• Make it morefluent. Provide some kind of builder syntax.
• Auto-generate themessage such that it never goes out of sync with the test.

Enter JUnitAssert

JUnit ships with a class which is simply calledAssert. This class consists of a series of static methods that should help the user in writing concise assertions. Most of the methods have the following shape:

publicstaticvoidassertXYZ(message,expectedValue,actualValue){/* ... */}

... wheremessage is an optional string that is printed instead of the auto-generated message if the assertion fails. Tests in this fashion look like this:

importorg.junit.*publicclassPersonTest{@TestpublicvoidtestWithAssert(){Personp=newPerson();Assert.assertNull("A new Person should not have an initial name.",p.getName());p.setName("John Doe");Assert.assertEquals("John Doe",p.getName());}}

In addition, one would typically useimport static org.junit.Assert.* to statically import the assertion methods. This way, theAssert. in front of an assertion can be omitted.

This is already a big step forward from the initialif construction: it fits in one line. However, there are several problems with this approach:

• It is really REALLY easy to mess up the assertion call and swapexpected andactual.
• As themessage is always the first parameter, and this parameter also happens to be optional, it is not always immediately clear if the first passed string is theexpected value or themessage.
• The auto-generated assertion error messages were rather basic, and the number of available assertion methods was quite limited.

Hamcrest! Wait... what?

Several years after JUnit was released, a nifty little library by the (very odd) name ofHamcrest was released. Hamcrest was built around the idea that an assertion is essentially amatch criterion, and as such it should be represented by aMatcher object. This allows for a greater variation of assertions with better error messages and a more fluent syntax:

@TestpublicvoidhamcrestTest(){Personp=newPerson();assertThat(p.getHobbies(),is(empty());assertThat(p.getName(),is(nullValue());p.setName("John Doe");p.setBirthDate(newDate());Set<String>newHobbies=newHashSet<>();newHobbies.add("coding");newHobbies.add("blogging");newHobbies.add("reading");p.setHobbies(newHobbies);assertThat(p.getName(),is("John Doe");assertThat(p.getHobbies(),not(hasItem("programming"));assertThat(p.getHobbies(),containsInAnyOrder("coding","blogging","programming");assertThat(p.getHobbies().size(),is(3));assertThat(p.getBirthDate().getTime(),is(greaterThan(0L));}

As you can see, theassertThat(value, matcher) method is the entry point. UnlikeassertEquals, it is perfectly clear which is theexpected and which is theactual value, so that's a big plus right out of the gate. A downside is that, due to the fact thatassertThat(...) has so many different overloads, you cannot useasserThat(p.getName(), is(null)), becausenull makes it ambiguous which override to use. Instead, you need to usenullValue(), which is essentially just a matcher that checks for equality withnull.

Hamcrest also introduced negated conditions withnot(...), easy numeric comparisons, as well as helpers for collections. All of these (in particular the collection helpers) generate quite useful error messages on their own right, so providing a custom error message (while possible) is usually not necessary anymore.

The downside of Hamcrest is that it relies heavily on static imports, which may even cause import conflicts (if you also use static methods a lot internally). Another drawback is that, while the assertion lines are now fluent to read, they are not actually very fluent to write:

// alright, let's test the name...p.getName()|// ... oh, I forgot the assertThat...assertThat(|p.getName()// ... moves carret all the way back again...assertThat(p.getName(),|// ok IDE, I need the "is" method, do your magic!assertThat(p.getName(),is|// (200 completion proposals show up)// *sigh*assertThat(p.getName(),is("John Doe"));|// finally!

See what I mean? Luckily, some folks out there were hitting the same issues.

Truth be told!

Truth is a library similar to Hamcrest, except that it offers a fluent builder API:

@TestpublicvoidtestTruth(){Personp=newPerson();assertThat(p.getName()).isNull();Set<String>newHobbies=newHashSet<>();newHobbies.add("coding");newHobbies.add("blogging");newHobbies.add("reading");p.setHobbies(newHobbies);assertThat(p).containsExactly("coding","blogging","reading");}

This is now finally a fluent API for tests, and one that also doesn't require too many static imports. Just follow the code completion of your IDE and you will create powerful assertions in no time.

As always, there are some caveats here as well. My most pressing concern with this solution is that, in comparison to Hamcrest, it is very difficult to extend. The Truth library itself is not under your control (unless you fork it...), so you cannot simply add new methods to existing classes to do your custom assertions.

Testing how itshouldBe

We now leave the safe haven of Java and venture out into the wild. As it turns out, Kotlin lends itself well to build a testing mini-framework which consists of only two functions (and the Hamcrest library):

infixfun<T>T.shouldBe(other:T):Unit{assertThat(this,is(other));}infixfun<T>T.shouldBe(matcher:Matcher<T>):Unit{assertThat(this,matcher);}

"How does this help" and "what in blazes is this" you may ask. Well, we are defining twoextension functions that reside onObject (or, in Kotlin:Any). Which means: we can now callx.shouldBe(...) on anything, no matter whatx is. In addition, it is aninfix function, which means that we can drop the dot, the opening and the closing brace.

Check it out:

@Testfun testKotlin(){    Person p = Person()    p.name shouldBe null    p.name = "John Doe"    p.name shouldBe "John Doe"    p.hobbies = setOf("coding", "blogging", "reading")    p.hobbies.size shouldBe 3    p.birthDate = Date()    p.birthDate!!.time shouldBe greaterThan(0L)}

Nowthis is the kind of readability that I am looking for!

Further Reading

If you feeling adventurous, I also recommend taking a look at

• Spock (Groovy Testing)
• Cucumber (Origin of the Gherkin test syntax)

Closing Words

I hope you enjoyed this little excursion to assertion libraries. Feel free to share your experiences and provide recommendations on libraries and/or coding styles that I have missed!