This document describes the structure of the instrumentation samples providedfor theGo,Java,Node.js, andPython languages.These samples provide guidance about how to instrument an application to use theOpenTelemetry SDK and an OpenTelemetry collector.

The instrumentation in these samples, which includes using the OpenTelemetry SDKand the SDK's in-process OTLP exporter, is vendor neutral. Thein-process exporter sends telemetry to the OpenTelemetry collector, which receivesthat data and then sends it to your Google Cloud project. The collector containsthe binding to Google Cloud. These samples use Google Cloud exporters to sendlog and metric data to your project. However, they send trace data to yourproject by using the Telemetry API.

You might be interested in other samples that illustrate differentconfigurations:

• Migrate from the Trace exporter to the OTLP endpointdescribes how to use in-process instrumentation to send trace data directly toyour Google Cloud project.

  We recommend that you use an OpenTelemetry collector to export your telemetrydata when your environment supports use of a collector. If you can't use acollector, then you must use an in-process exporter that directly sends datato your Google Cloud project.

• Correlate metrics and traces by using exemplarsdescribes how to configure a Go application to generate exemplars.Anexemplar is an example data point attached to a metric data point.You can use exemplars to correlate your trace and metric data.

• Use the Ops Agent and OpenTelemetry Protocol (OTLP)describes how you can configure the Ops Agent and an OTLP receiver tocollect metrics and traces from an application.

The samples for Go, Java, Node.js, and Python use theOpenTelemetry protocol to collect trace and metric data.The samples configure a logging framework to writestructured logs and theOpenTelemetry collector is configured to read fromthe application'sstdout stream. For framework recommendations, seeChoose an instrumentation approach.

The applications are built and deployed by using Docker. You don't have touse Docker when you instrument an application with OpenTelemetry.

You can run the samples in the Cloud Shell, on Google Cloudresources, or on a local development environment.

Deep dive

The samples use theOpenTelemetry Collector as a sidecarto receive and enrich the application's telemetry, which is then sent to yourGoogle Cloud project by using aGoogle Cloud exporter. The exporterconverts thetelemetry into a format compatible with the Cloud Trace API, Cloud Monitoring API,or Cloud Logging API. Next, they send the transformed data to yourGoogle Cloud project by issuing an API command.

The samples show how to do the following:

1. Configure OpenTelemetry to collect metrics and traces by using theOpenTelemetry collector.

   If you review the samples, you'll notice that the complexity of this stepis language dependent. For example, for Go, this step configures themain function to call a function that configures the collection ofmetrics and traces. For Go, the HTTP server and client are also updated.

2. Configure a logging framework to writestructured logs.

   We recommend that your applications write structured logs, which resultsin the log payload being formatted as a JSON object. For these logs, you canconstruct queries that search specific JSON paths and you canindex specific fields in the log payload.

   Some services, like Google Kubernetes Engine, have built-in agents that scrapestructured logs and send those logs to your Google Cloud project. Otherservices, like Compute Engine, require that you install an agent, whichscrapes and sends your logs. If you want to learn about agents you install,seeOps Agent overview.

   You don't need to install any agents to use these samples.

3. Configure Docker files. All samples contain the following yaml files:

   • docker-compose.yaml: Configures the services for the application,theOpenTelemetry collector, and a load generator.For example, the service for the OpenTelemetry collector,otelcol,specifies an image, a volume, and environment variables.The endpoint for the OpenTelemetry collector is set by theOTEL_EXPORTER_OTLP_ENDPOINT environment variable, which isspecified in theapp service.

   • otel-collector-config.yaml: Configures the OpenTelemetry collector. Thiscollector specifies receivers, exporters, processors, and pipelines.

     Thetelemetry service defines pipelines for trace, metric, and logdata. Each pipeline entry specifies a receiver, a processor, and anexporter. The same receiver,otlp, is used for metrics and traces.

     Theexporters section describes how collected data is exported toa Google Cloud project. For log and metric data,Google Cloud exporters are used. These exporters convertthe telemetry into a format compatible with the corresponding APIand then send the transformed data to yourGoogle Cloud project by issuing an API command.In contrast, trace data is sent to your project by using theTelemetry API, which supports OTLP.

   • docker-compose.creds.yaml: This file optionally mounts aGoogle Cloud credentials file in theotelcol container. This fileis needed when a sample is run on a local machine where theApplication Default Credentials (ADC) are availableonly as a file.

Required permissions

• To get the permissions that you need to for the sample applications to write log, metric, and trace data, ask your administrator to grant you the following IAM roles on your project:

  • Logs Writer (roles/logging.logWriter)
  • Monitoring Metric Writer (roles/monitoring.metricWriter)
  • Cloud Telemetry Traces Writer (roles/telemetry.tracesWriter)

  If you run the samples in the Cloud Shell, on Google Cloud resources,or on a local development environment, then the previous permissionsare sufficient to write log, metric, and trace data. For productionapplications, a service account typically provides the required credentials.

• To get the permissions that you need to view your log, metric, and trace data, ask your administrator to grant you the following IAM roles on your project:

  • Logs Viewer (roles/logging.viewer)
  • Monitoring Viewer (roles/monitoring.viewer)
  • Cloud Trace User (roles/cloudtrace.user)

  For more information about granting roles, seeManage access to projects, folders, and organizations.

  You might also be able to get the required permissions throughcustom roles or otherpredefined roles.

Required APIs

Enable the Cloud Logging, Cloud Monitoring, Cloud Trace, and Telemetry APIs:

gcloudservicesenablelogging.googleapis.com monitoring.googleapis.com cloudtrace.googleapis.com telemetry.googleapis.com

What's next

To learn more about collectors, seeGoogle-Built OpenTelemetry Collector.

Explore the samples that use collector-based exports.

• Go sample.
• Java sample.
• Node.js sample.
• Python sample.