Development Guidelines#

Logger Abstraction#

Apache Arrow Java uses the SLF4J API, so please configure SLF4J to see logs (e.g. via Logback/Apache Log4j):

  1. If no jar dependencies are added by the user via Logback or Apache Log4j then SLF4J will defaultto no-operation (NOP) logging.

  2. If a user adds any dependencies via Logback or Apache Log4j but does not configure/add/definelogback.xml/log4j2.xml, then logs will default to DEBUG mode.

  3. To disable debug logs, the user must define their own rules within their logback.xml/log4j2.xmland define their own loggers.

Unit Testing#

Unit tests are run by Maven during the build.

To speed up the build, you can skip them by passing -DskipTests.

$cdarrow/java$mvn\-Darrow.cpp.build.dir=../java-dist/lib-Parrow-jni\-Darrow.c.jni.dist.dir=../java-dist/lib-Parrow-c-data\cleaninstall

Performance Testing#

Thearrow-performance module contains benchmarks.

Let’s configure our environment to run performance tests:

In case you need to see your performance tests on the UI, then, configure (optional):

Lets execute benchmark tests:

$cdbenchmarks$conbenchjava-micro--help$conbenchjava-micro    --iterations=1    --commit=e90472e35b40f58b17d408438bb8de1641bfe6ef    --java-home=<absolute path to your java home>    --src=<absolute path to your arrow project>    --benchmark-filter=org.apache.arrow.adapter.AvroAdapterBenchmarks.testAvroToArrowBenchmark                              Mode  Cnt       Score   Error  UnitsAvroAdapterBenchmarks.testAvroToArrow  avgt       725545.783          ns/opTime to POST http://localhost:5000/api/login/ 0.14911699295043945Time to POST http://localhost:5000/api/benchmarks/ 0.06116318702697754

Then go to:http://127.0.0.1:5000/ to see reports:

UI Home:

../../_images/conbench_ui.png

UI Runs:

../../_images/conbench_runs.png

UI Benchmark:

../../_images/conbench_benchmark.png

Integration Testing#

Integration tests can be runvia Archery.For example, assuming you only built Arrow Java and want to run the IPCintegration tests, you would do:

$archeryintegration--run-ipc--with-java1

Code Style#

The current Java code follows theGoogle Java Style with Apache license headers.

Java code style is checked bySpotless during the build, and the continuous integration build will verifythat changes adhere to the style guide.

Automatically fixing code style issues#

  • You can check the style without building the project withmvnspotless:check.

  • You can autoformat the source withmvnspotless:apply.

Example:

Thefollowingfileshadformatviolations:src/main/java/org/apache/arrow/algorithm/rank/VectorRank.java@@-15,7+15,6@@·*·limitations·under·the·License.·*/-package·org.apache.arrow.algorithm.rank;import·java.util.stream.IntStream;Run'mvn spotless:apply'tofixtheseviolations.

Code Formatter for Intellij IDEA and Eclipse#

Follow the instructions to set up google-java-format for:

Checkstyle#

Checkstyle is also used for general linting. The configuration is located atcheckstyle.You can also just check the style without building the project.This checks the code style of all source code under the current directory or from within an individual module.

$mvncheckstyle:check

Mavenpom.xml style is enforced with Spotless usingApache Maven pom.xml guidelinesYou can also just check the style without building the project.This checks the style of all pom.xml files under the current directory or from within an individual module.

$mvnspotless:check

This applies the style to all pom.xml files under the current directory or from within an individual module.

$mvnspotless:apply

Build Caching#

Build caching is done through Develocity (formerly Maven Enterprise). To forcea build without the cache, run:

mvn clean install -Ddevelocity.cache.local.enabled=false -Ddevelocity.cache.remote.enabled=false

This can be useful to make sure you see all warnings from ErrorProne, for example.

ErrorProne#

ErrorProne should be disabled for generated code.