Repository files navigation

A Java version ofsimdjson - a JSON parser using SIMD instructions,based on the paperParsing Gigabytes of JSON per Secondby Geoff Langdale and Daniel Lemire.

byte[]json =loadTwitterJson();SimdJsonParserparser =newSimdJsonParser();JsonValuejsonValue =parser.parse(json,json.length);Iterator<JsonValue>tweets =jsonValue.get("statuses").arrayIterator();while (tweets.hasNext()) {JsonValuetweet =tweets.next();JsonValueuser =tweet.get("user");if (user.get("default_profile").asBoolean()) {System.out.println(user.get("screen_name").asString());    }}

byte[]json =loadTwitterJson();SimdJsonParserparser =newSimdJsonParser();SimdJsonTwittertwitter =simdJsonParser.parse(buffer,buffer.length,SimdJsonTwitter.class);for (SimdJsonStatusstatus :twitter.statuses()) {SimdJsonUseruser =status.user();if (user.default_profile()) {System.out.println(user.screen_name());    }}recordSimdJsonUser(booleandefault_profile,Stringscreen_name) {}recordSimdJsonStatus(SimdJsonUseruser) {}recordSimdJsonTwitter(List<SimdJsonStatus>statuses) {}

The library is available in theMaven Central Repository.To include it in your project, add the following dependency to thebuild.gradle file:

implementation("org.simdjson:simdjson-java:0.1.0")

or to thepom.xml file:

<dependency>    <groupId>org.simdjson</groupId>    <artifactId>simdjson-java</artifactId>    <version>0.1.0</version></dependency>

Please remember about specifying the desired version.

Note that simdjson-java follows theSemVer specification, which means, for example, that a majorversion of zero indicates initial development, so the library's API should not be considered stable.

We require Java 24 or better.

To run the JMH benchmarks, execute the following command:

./gradlew jmh

To run the tests, execute the following command:

./gradlew test

This section presents a performance comparison of different JSON parsers available as Java libraries. The benchmark usedthetwitter.json dataset, and its goal was to measure the throughput (ops/s) of parsingand finding all unique users with a default profile.

Environment:

• CPU: Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
• OS: Ubuntu 24.04 LTS, kernel 6.8.0-1008-aws
• Java: OpenJDK 64-Bit Server VM (build 21.0.3+9-Ubuntu-1ubuntu1, mixed mode, sharing)

DOM parsers (ParseAndSelectBenchmark):

Schema-based parsers (SchemaBasedParseAndSelectBenchmark):

Environment:

• CPU: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
• OS: Ubuntu 24.04 LTS, kernel 6.8.0-1008-aws
• Java: OpenJDK 64-Bit Server VM (build 21.0.3+9-Ubuntu-1ubuntu1, mixed mode, sharing)

DOM parsers (ParseAndSelectBenchmark):

Schema-based parsers (SchemaBasedParseAndSelectBenchmark):

To reproduce the benchmark results, execute the following command:

./gradlew jmh -Pjmh.includes='.*ParseAndSelectBenchmark.*'

The benchmark may take several minutes. Remember that you need Java 18 or better.