Repository files navigation

This project contains a set of performance micro-benchmarks, based on excellentJMH package.Benchmarks exercise JSON reading and/or writing performance,using widely-used, popular Java JSON libraries like:

• GSON
• Jackson
  • Also: lighter-weightJackson-jr

as well as some of newer Java JSON library (or multi-format with JSON support) options:

• Boon
• DSL-JSON (compatible withDSL-Platform)
• Johnzon
  • pre-1.0, incubation release
• json-io
• json-parse
• Juneau
• Moshi: Android-optimized lib (by some of GSON authors)

The criteria for inclusion here is that for a library to be included it should

1. be published to the central Maven repository (so we can include official builds)
2. be able to read and write POJOs, not just "Lists and Maps" or "custom tree nodes library defines" (although some tests may also exercise these styles as well)

and for this reason some commonly used libraries (like old "org.json" library and "simple-json") are not included.

To run the tests, you will first need to build the test jar which contains all test code as well as libraries being tested. This can be done by:

mvn clean install

after this, tests are run the wayjmh tests are, by just running "executable" jar with:

java [jvm-options] -jar target/microbenchmarks.jar [options] [test-regexp]

for example:

java -Xmx256m -jar target/microbenchmarks.jar -wi 4 -i 5 -f 9 ".*DZoneReadPojo.*read10FromStream" java -Xmx256m -jar target/microbenchmarks.jar -wi 4 -i 5 -f 9 ".*DZoneWrite.*write10UsingStream"

both of which would run the "DZone" write test with 10 items, using 9 iterations of 5 seconds, with warmup time of 4 seconds; first test for read (JSON into POJO) and second write (POJO to JSON) performance.

All options are explained by jmh documentation; an easy way to see options available is to enter:

java -jar target/microbenchmarks.jar -h

Following tests were written inspired by a [DZone Java Performance](https://t.co/10lR0tQJjV] article.The original tests had many unfortunate problems; starting with the fact that it does not perform proper warmup, nor run long enough to give statistically meaning results.One can also argue whether serialization as String is a meanginful test (since it is rarely used for production), but that test case is included as-is along with other options.

Test as implemented here relies onjmh to provide proper performance test setup, measurements, warmup, and to avoid common gotchas that plague naive Java performance tests.If not done so yet, you may want to read the longer explanation ofreasons to use JMH.And for common problems with Java/JSON performance testing, you may want to readOn proper performance testing of Java JSON processing.

There are 3 flavors of the test for writing out a List of 10, 1000 and 100,000 items of typeMeasurementRecord:

• write10AsString (andwrite1kAsString,write100kAsString): serialization asjava.lang.String
• write10UsingWriter (write1kUsingWriter,write100kUsingWriter): serialization using a "do-nothing"java.io.Writer
• write10UsingStream (write1kUsingStream,write100kUsingStream): serialization using a "do-nothing"java.io.OutputStream

the idea being that different types of output incur different kinds and amounts of overhead.For example, aggregating output as ajava.lang.String requires much more memory allocation and (indirectly) more Garbage Collection for discarded JSON Strings.Conversely libraries are optimized differently for different types of output targets; some implement native output for all targets and others only have one basic target.

Size of each POJO is quite small; field names are long and values mostly numbers.This may not be the most commonly found kind of content, but was used by the original set upand is used here unmodified.

To run the test with a List of 10 items, serializing results as a Java String, you could use:

java -Xmx256m -jar target/microbenchmarks.jar ".*DZoneWrite.*write10AsString" -wi 4 -i 5 -f 9

or 1000 items intoOutputStream

java -Xmx256m -jar target/microbenchmarks.jar ".*DZoneWrite.*write1kUsingStream" -wi 4 -i 5 -f 9

Similar to writing, there are similar variations for reading JSON as POJOs.These tests are named likeDZoneReadPojoJackson:

• read10FromString (andread1kFromString,read100kFromString): deserialization fromjava.lang.String
• read10FromBytes (read1kFromBytes,read100kFromBytes): deserialization from givenbyte[]

But in addition, there are also alternatives for reading same JSON as "untyped" values; that is, asjava.util.Maps,java.util.Lists,Strings,Numbers andBooleans:

• read10FromString (andread1kFromString,read100kFromString): deserialization fromjava.lang.String
• read10FromBytes (read1kFromBytes,read100kFromBytes): deserialization from givenbyte[]

and the difference is from naming tests classes likeDZoneReadMapJackson (replacingPOJO withMap); test names are the same.

SeeWiki for sample results.