This document provides an overview of logging and troubleshootingcommon issues with Cloud Client Libraries for Java.

Logging

There are two ways to enable logging for the client libraries:usingjava.util.logging, or using Client Library Debug Logging withSLF4J.

Usingjava.util.logging

Cloud Client Libraries for Java uses the Java logging API (java.util.logging)package. Configuring the logging level reveals information that helps youtroubleshoot, including:

• Timing of underlying client-server communication.
• Request and response message headers.
• Verbose messages in underlying dependency libraries.

To quickly enable verbose logging for Cloud Client Libraries for Java, createa filelogging.properties with the following content:

# run java program pointing to this properties file with the java arg#   -Djava.util.logging.config.file=path/to/logging.propertieshandlers=java.util.logging.ConsoleHandlerjava.util.logging.SimpleFormatter.format=%1$tF %1$tT,%1$tL %4$-8s %3$-50s - %5$s %6$s%n# --- ConsoleHandler ---java.util.logging.ConsoleHandler.level=ALLjava.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter.level=INFO# --- Specify logging level for certain packages ---# com.google.api is for HTTP 1.1 layercom.google.api.level=ALL# io.grpc is for gRPC + Netty layerio.grpc.level=FINE# com.google.auth is for authenticationcom.google.auth.level=FINE# Example when you specify the storage library's level. This works when# the target Cloud library uses the logging API.com.google.cloud.storage.level=INFO

Next, run your application with-Djava.util.logging.config.file=path/to/logging.propertiesas theVM argument, not theProgram argument.

If you use IntelliJ, you specify the VM argument inRun/Debug Configuration:

If the JVM of your program is running with the configuration correctly, you seethe FINE-level logging in your console.

Example output:

2023-04-05 13:03:01,761 FINE     com.google.auth.oauth2.DefaultCredentialsProvider  - Attempting to load credentials from well known file: /usr/local/google/home/suztomo/.config/gcloud/application_default_credentials.json2023-04-05 13:03:01,847 FINE     io.grpc.ManagedChannelRegistry                     - Unable to find OkHttpChannelProviderjava.lang.ClassNotFoundException: io.grpc.okhttp.OkHttpChannelProvider    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)    at java.base/java.lang.Class.forName0(Native Method)    at java.base/java.lang.Class.forName(Class.java:315)...

This is one way to configure logging level. For more information about Javalogging API usage, seeJava Logging Overview.

Client Library Debug Logging (SLF4J)

Cloud Client Libraries comes with opt-in Debug Logging that can help youtroubleshoot your application's integration with the API.When logging is activated, key events such as requests and responses,along with data payloads and metadata, such as headers, are logged to thestandard error stream.

WARNING: Client Library Debug Logging includes your data payloads inplaintext, which could include sensitive data such as PII for yourself or yourcustomers, private keys, or other security data that could be compromised ifleaked.Always practice good data hygiene with your application logs, and follow theprinciple of least access. Google also recommends that client library debuglogging be enabled only temporarily during active debugging, and not usedpermanently in production.

Prerequisite

Our libraries support debug logging using theSLF4Jinterface.

Client libraries and Libraries BOM does not include slf4j-api dependency.You will need to set up logging dependencies including SLF4J and correspondinglogging implementations and configurations before turning on client librarydebug logging.

NOTE: You need to have SLF4J and corresponding logging providers, e.g.,logback, log4j2, etc., in your classpath in order to use the client librarydebug logging feature; otherwise, no debug logging even if you enable it usingenvironment variable.

Enable Logging with an environment variable

Debug logging is turned off by default.Debug logging is not turned on unless explicitly even if the prerequisites aremet.

You can enable client library debug logging by setting the environment variableGOOGLE_SDK_JAVA_LOGGING totrue, case insensitive.If unset or any other value, then client library debug logging is disabled.

Example logging setups

The documentation provides an example of debug logging using Logback.

Add Logback dependencies to your application, this will bring in slf4jdependency transitively.Users should refer to theLogback setupfor latest versions.You can skip this step if your application already has logback dependencies in the classpath.

If you are using Maven:

<dependency><groupId>ch.qos.logback</groupId><artifactId>logback-classic</artifactId><version>LATEST_VERSION</version></dependency>

Add the content to the logback configuration file if you want to see DEBUG level logs.You can skip this step if you only need INFO level logs.Refer to theLogback configurationfor more information.

<!--setClientLibrarylogleveltoDEBUG--><loggername="com.google.api"level="DEBUG"/><!--setAuthLibrarylogleveltoDEBUG--><loggername="com.google.auth"level="DEBUG"/>

ALPN is not configured properly

If you see exceptions related toALPN is not configured properly, such as:

Caused by: java.lang.IllegalArgumentException: ALPN is not configured properly. See https://github.com/grpc/grpc-java/blob/master/SECURITY.md#troubleshooting for more information.

Use thecompatibility checker to see if your environment is compatible with gRPC-based clients.

Incompatibility might mean one of the following:

• You are not on asupported platform.
• There are classpath conflicts withnetty.
• You are seeing any of the conflicts specified ingRPC Troubleshooting guide.

TroubleshootClassNotFoundException,NoSuchMethodError, andNoClassDefFoundError

These errors are often caused by having multiple versions or conflictingversions of the same dependency in the classpath. These dependencyconflicts often occur withguava orprotobuf-java.

Multiple sources might cause classpath conflicts:

• Multiple versions of the same transitive dependency in the dependency tree.
• Your runtime classpath has different versions of dependencies than what youspecified in the build.

For example, if you have a direct or a transitive dependency on Guava version19.0, andgoogle-cloud-java uses Guava version 30.0, thengoogle-cloud-javamight be using Guava methods that don't exist in Guava 19.0, and causeNoSuchMethodError.

Similarly, if your classpath has an older version ofprotobuf-java,butgoogle-cloud-java requires a later version, then you might seeNoClassDefFoundError that fails to initializegoogle-cloud-java classes.

For example:

java.lang.NoClassDefFoundError: Could not initialize class com.google.pubsub.v1.PubsubMessage$AttributesDefaultEntryHolder

Validate the conflict

Check the dependency tree to see if you have multiple versions of the samedependencies:

$ mvn dependency:tree

Look for versions of potentially conflicting dependencies such asguava orprotobuf-java.

If you experience the error only during runtime, then your runtime environmentmight be introducing conflicting JARs into your runtime classpath. A typicalcase is that Hadoop, Spark, or other server software that your applicationruns on has conflicting versionsnetty,guava, orprotobuf-java JARs inthe classpath.

Detect conflicts during build

To detect dependency linkage errors at compile time, add theLinkage Checker Enforcer Rulein your pom.xml:

      <plugin>        <groupId>org.apache.maven.plugins</groupId>        <artifactId>maven-enforcer-plugin</artifactId>        <version>3.0.0-M3</version>        <dependencies>          <dependency>            <groupId>com.google.cloud.tools</groupId>            <artifactId>linkage-checker-enforcer-rules</artifactId>            <version>1.5.7</version>          </dependency>        </dependencies>        <executions>          <execution>            <id>enforce-linkage-checker</id>            <!-- Important! Should run after compile -->            <phase>verify</phase>            <goals>              <goal>enforce</goal>            </goals>            <configuration>              <rules>                <LinkageCheckerRule                    implementation="com.google.cloud.tools.dependencies.enforcer.LinkageCheckerRule"/>              </rules>            </configuration>          </execution>        </executions>      </plugin>

However, there is no way to detect runtime classpath conflicts. You must befully aware of which JARs/classes are included in the runtime classpathbecause every server environment is different.

Resolve the conflict

There are different strategies to resolve conflicts, but you must understandthe root cause of the conflicts:

• If you control the dependency tree, upgrade the conflicting dependencies (forexample, upgrading Guava). This approach is the most robust, but it can requiresignificant effort and multiple library releases to ensure compatibility.
• If you can't modify and push new versions of your dependencies, importcom.google.cloud:libraries-bom:25.1.0 (or a more recent version) to selectconsistent dependency versions. This approach simplifies dependencymanagement. For example, this is how you can depend on consistent versions ofGuava andcom.google.cloud:google-cloud-storage without explicitly settingthe version of either one:

...<dependencyManagement><dependencies><dependency><groupId>com.google.cloud</groupId><artifactId>libraries-bom</artifactId><version>25.1.0</version><type>pom</type><scope>import</scope></dependency></dependencies></dependencyManagement>...<dependencies><dependency><groupId>com.google.cloud</groupId><artifactId>google-cloud-storage</artifactId></dependency><dependency><groupId>com.google.guava</groupId><artifactId>guava</artifactId></dependency>...</dependencies>...

• The libraries-bom release notes show the compatible dependencylibraries. For example, https://github.com/googleapis/java-cloud-bom/releases/tag/v26.31.0shows:

  These client libraries are built with the following Java libraries:- Guava: 32.1.3-jre- Protobuf Java: 3.25.2- Google Auth Library: 1.22.0- Google API Client: 2.2.0- gRPC: 1.61.0- GAX: 2.41.0- Google Cloud Core: 2.31.0

  By examining the dependency graph of your project(mvn dependency:tree -Dverbose,gradle dependencies, orsbt dependencyTree), you might find some dependencies have unexpectedversions, which might cause dependency conflicts.

• If changing dependency versions introduces other failures,considershading dependenciesthat conflict with Google Cloud Java libraries.

  For example, to shadeguava andprotobuf-java:

  <plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-shade-plugin</artifactId><version>...</version><executions><execution><phase>package</phase><goals><goal>shade</goal></goals><configuration><keepDependenciesWithProvidedScope>false</keepDependenciesWithProvidedScope><relocations><!--moveprotobuftoashadedpackage--><relocation><pattern>com.google.protobuf</pattern><shadedPattern>myapp.shaded.com.google.protobuf</shadedPattern></relocation><!--moveGuavatoashadedpackage--><relocation><pattern>com.google.common</pattern><shadedPattern>myapp.shaded.com.google.common</shadedPattern></relocation></relocations></configuration></execution></executions></plugin>