A while back I was confronted with a part of a legacy system that generates PDF documents. This legacy system used a well known library for generating the requested PDF files. The good news was that there were a decent amount of tests available. The not so good news was that these tests made heavy use of test doubles for swapping out most of the types provided by the third-party API. I strongly believe that using test doubles in such cases is not a good choice. In the past I already wrote about why toavoid using test doubles for types that you don’t own.

Tests like these might become a large impediment whenever we upgrade to a newer version of the third-party library. Major versions quite often introduce breaking changes to existing API’s. Whenever tests are strongly coupled to such an API, they basically need to be rewritten during the upgrade.

In order to reduce the coupling of these tests, I decided to replace them withApproval Tests instead. This technique is also known as“Golden Master” or“Characterization Test”. The idea behind anApproval Test is that after the first test run, some output needs to be visually verified and approved. During subsequent test runs, the approved output will be compared to current output. When there’s a difference in the output, the test will fail.

This is especially useful whenever we have to deal with code of a legacy system. You can check out this video from Emily Bache where she demonstrates theGilded Rose refactoring kata using Approval Tests. Highly recommended!

Usually the output ofApproval Tests is captured in plain text files, which has nothing to do with PDF files. So I decided to extend theJava Version of an Approval Test libraryto support PDF documents as well. Let’s have a look at some example code to demonstrate this extension.

Suppose that we have a small application that generates a PDF document. A generated document contains the refrain of a well-known song lyric. The following code shows a possible implementation.

publicclassSingAlongPdfGenerator{publicByteArrayOutputStreamgenerate(SingAlongDatapdfData){varoutputStream=newByteArrayOutputStream();PdfDocumentpdf=newPdfDocument(newPdfWriter(outputStream));Documentdocument=newDocument(pdf);document.add(newParagraph("Let's sing-a-long:"));Listlist=newList().setSymbolIndent(12).setListSymbol("\u2022");for(varrefrainLine:pdfData.getRefrainLines()){list.add(newListItem(refrainLine));}document.add(list);document.close();returnoutputStream;}}publicclassSingAlongData{privatefinalList<String>refrainLines;publicSingAlongData(List<String>refrainLines){this.refrainLines=refrainLines;}publicList<String>getRefrainLines(){returnrefrainLines;}}

TheSingAlongPdfGenerator class provides a method namedgenerate that accepts aSingAlongData instance as its only parameter. TheSingAlongData class is merely a DTO that provides a list of refrain lines. Thegeneratemethod creates a new document containing a paragraph of text and a list of the refrain lines.

Let’s have a look at the code of the correspondingApproval Test.

publicclassSingAlongPdfGeneratorTests{@TestpublicvoidgenerateRickRollPdf(){varrefrainLines=Arrays.asList("Never gonna give you up","Never gonna let you down","Never gonna run around and desert you","Never gonna make you cry","Never gonna say goodbye","Never gonna tell a lie and hurt you");vardata=newSingAlongData(refrainLines);varpdfGenerator=newSingAlongPdfGenerator();varresult=pdfGenerator.generate(data);PdfApprovals.verify(result);}}

First we create an instance of theSingAlongData DTO, specifying some test data. Next we create an instance of the Subject Under Test, which in this case is theSingAlongPdfGenerator class and call thegenerate method. This returns aByteArrayOutputStream containing the data of the PDF document. Then we verify the result by calling thePdfApprovals.verify method. Notice that the anatomy of anApproval Test is identical to any other type of test as it also adheres to theArrange, Act, Assert pattern.

A newApproval Test always fails the very first time that it gets executed. After the initial test run, a PDF file is generated that needs to be visually verified and approved. So when we first run thegenerateRickRollPdf test, a file with the nameSingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf appears which has the following content:

After we verified the PDF document, we approve it using the following command:

mv ~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf ~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf

At this point we have a PDF file namedSingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf. When we now execute our test again, it passes as the newly generated PDF document matches the approved PDF document.

Note that we also have to make sure to commit the approved PDF file alongside our test code. Otherwise we have to approve the output of the test again when executed on another machine.

Let’s say that we want to make a change to the implementation of ourSingAlongPdfGenerator class. For example, we’re going to make a small change to the text inside the paragraph.

document.add(newParagraph("Let's sing-a-long shall we?"));

When we execute our test again, it fails due to the change that we’ve made.

Failed Approval  Approved:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf  Received:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf

The test created a “received” PDF file again alongside the existing PDF file that we’ve approved earlier. We now have to visually compare the received and the approved file side-by-side. Needless to mention that this is going to be quite cumbersome. For this reason I’ve added the capability that in case of failing test, a third PDF file is generated that contains the annotated differences between the two PDF files.

The purple markers indicate where the changes are located in the document. The green colon indicates what is expected (approved). The red text that we’ve added to the paragraph shows the actual text found in the document (received). We now have to decide whether we want to approve the changes that we’ve made or not. Let’s say that we are happy with the change that we’ve made. To do that we simply run the approval command again as shown earlier. And that’s it.

With just a handful of theseApproval Tests I was able to eliminate all the tightly coupled tests. Generating PDF files is more often than not an infrastructure concern.Sociable tests tests are much more appropriate in this case compared tosolitary tests. UsingApproval Tests this way turned out to be a very valuable approach.