Set up a gRPC application on Google Cloud with Microservices observability
Microservices observability tools provide you with the ability toinstrument your applications to collect and present telemetry data inCloud Monitoring, Cloud Logging, and Cloud Trace from gRPC workloadsdeployed on Google Cloud and elsewhere.Microservices observability works with any deployment that has beengranted permission to access Monitoring, Logging,and Trace by enabling the Microservices API.
In this tutorial you learn how to use Microservices observabilityfeatures by building a simple gRPC application on Google Cloud using Compute Engine and instrument your application withMicroservices observability, andview them come up live on Monitoring and Logging.
Objectives
At a high level, you do the following.
As a service developer, you can:
- Create a gRPC application with the language of your choice (C++, Go, or Java).
- Opt in your application and control the Microservices observability plugin.
- Deploy the application on a Compute Engine VM.
As a service operator, you consume the collected data in various ways:
- View traces on Trace.
- View metrics on the monitoring dashboard called Microservices (gRPC) Monitoring.
- View metrics on Metrics Explorer.
- Inspect log entries on Logs Explorer.
Costs
In this document, you use the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage, use thepricing calculator.
When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, seeClean up.
Before you begin
Console
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
- Create a project: To create a project, you need the Project Creator (
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission.Learn how to grant roles.
Verify that billing is enabled for your Google Cloud project.
Enable the Compute Engine, and Microservices API APIs.
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (
roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enablepermission.Learn how to grant roles.Create a service account:
- Ensure that you have the Create Service Accounts IAM role (
roles/iam.serviceAccountCreator).Learn how to grant roles. In the Google Cloud console, go to theCreate service account page.
Go to Create service account- Select your project.
In theService account name field, enter a name. The Google Cloud console fills in theService account ID field based on this name.
In theService account description field, enter a description. For example,
Service account for quickstart.- ClickCreate and continue.
Grant the following roles to the service account:Logging > Logs Viewer > Logs Writer, Monitoring > Monitoring Editor > Metrics Writer, Trace > Trace Admin > Trace Agent.
To grant a role, find theSelect a role list, then select the role.
To grant additional roles, clickAdd another role and add each additional role.
Note: TheRole field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later.- ClickContinue.
In theService account users role field, enter the identifier for the principal that will attach the service account to other resources, such as Compute Engine instances.
This is typically the email address for a Google Account.
ClickDone to finish creating the service account.
- Ensure that you have the Create Service Accounts IAM role (
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
- Create a project: To create a project, you need the Project Creator (
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission.Learn how to grant roles.
Verify that billing is enabled for your Google Cloud project.
Enable the Compute Engine, and Microservices API APIs.
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (
roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enablepermission.Learn how to grant roles.Create a service account:
- Ensure that you have the Create Service Accounts IAM role (
roles/iam.serviceAccountCreator).Learn how to grant roles. In the Google Cloud console, go to theCreate service account page.
Go to Create service account- Select your project.
In theService account name field, enter a name. The Google Cloud console fills in theService account ID field based on this name.
In theService account description field, enter a description. For example,
Service account for quickstart.- ClickCreate and continue.
Grant the following roles to the service account:Logging > Logs Viewer > Logs Writer, Monitoring > Monitoring Editor > Metrics Writer, Trace > Trace Admin > Trace Agent.
To grant a role, find theSelect a role list, then select the role.
To grant additional roles, clickAdd another role and add each additional role.
Note: TheRole field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later.- ClickContinue.
In theService account users role field, enter the identifier for the principal that will attach the service account to other resources, such as Compute Engine instances.
This is typically the email address for a Google Account.
ClickDone to finish creating the service account.
- Ensure that you have the Create Service Accounts IAM role (
- Read theMicroservices observability overview.
- Read about thetwo supported environment variables, decide which to use, and determine the values required by the environment variable.
cli
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
- Create a project: To create a project, you need the Project Creator (
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission.Learn how to grant roles.
Verify that billing is enabled for your Google Cloud project.
Enable the Compute Engine, and Microservices API APIs.
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (
roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enablepermission.Learn how to grant roles.Create a service account:
- Ensure that you have the Create Service Accounts IAM role (
roles/iam.serviceAccountCreator).Learn how to grant roles. In the Google Cloud console, go to theCreate service account page.
Go to Create service account- Select your project.
In theService account name field, enter a name. The Google Cloud console fills in theService account ID field based on this name.
In theService account description field, enter a description. For example,
Service account for quickstart.- ClickCreate and continue.
Grant the following roles to the service account:roles/logging.logWriter, roles/monitoring.metricWriter, roles/cloudtrace.agent.
To grant a role, find theSelect a role list, then select the role.
To grant additional roles, clickAdd another role and add each additional role.
Note: TheRole field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later.- ClickContinue.
In theService account users role field, enter the identifier for the principal that will attach the service account to other resources, such as Compute Engine instances.
This is typically the email address for a Google Account.
ClickDone to finish creating the service account.
- Ensure that you have the Create Service Accounts IAM role (
Install the Google Cloud CLI.
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
Toinitialize the gcloud CLI, run the following command:
gcloudinit
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
- Create a project: To create a project, you need the Project Creator (
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission.Learn how to grant roles.
Verify that billing is enabled for your Google Cloud project.
Enable the Compute Engine, and Microservices API APIs.
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (
roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enablepermission.Learn how to grant roles.Create a service account:
- Ensure that you have the Create Service Accounts IAM role (
roles/iam.serviceAccountCreator).Learn how to grant roles. In the Google Cloud console, go to theCreate service account page.
Go to Create service account- Select your project.
In theService account name field, enter a name. The Google Cloud console fills in theService account ID field based on this name.
In theService account description field, enter a description. For example,
Service account for quickstart.- ClickCreate and continue.
Grant the following roles to the service account:roles/logging.logWriter, roles/monitoring.metricWriter, roles/cloudtrace.agent.
To grant a role, find theSelect a role list, then select the role.
To grant additional roles, clickAdd another role and add each additional role.
Note: TheRole field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later.- ClickContinue.
In theService account users role field, enter the identifier for the principal that will attach the service account to other resources, such as Compute Engine instances.
This is typically the email address for a Google Account.
ClickDone to finish creating the service account.
- Ensure that you have the Create Service Accounts IAM role (
Install the Google Cloud CLI.
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
Toinitialize the gcloud CLI, run the following command:
gcloudinit
- Read theMicroservices observability overview.
- Read about thetwo supported environment variables, decide which to use, and determine the values required by the environment variable.
Create and connect to a Compute Engine VM
Use these instructions to create and connect to a Compute Engine VMinstance. On the VM you deploy your application and then instrument theapplication with Microservices observability.
Create a VM instance:
gcloud compute instances create grpc-observability-vm \ --image-family=debian-11 \ --image-project=debian-cloud \ --service-account=SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com
Connect to the VM instance:
gcloud compute ssh --project=$PROJECT_ID grpc-observability-vm
Deploy your application to the Compute Engine VM
You can either deploy an application of your choice to theCompute Engine VM that you created in the previous step, then skipthis step, or you can use an example to continue with instructions inyour preferred language.
C++
After you connect to the VM instance, run the following command.
sudo apt-get update -ysudo apt-get install -y git build-essential clanggit clone -b v1.54.0 https://github.com/grpc/grpc.git --depth=1
Go
Make sure that you have Go installed.
sudo apt-get install -y gitsudo apt install wgetwget https://go.dev/dl/go1.20.2.linux-amd64.tar.gzsudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf \go1.20.2.linux-amd64.tar.gzexport PATH=$PATH:/usr/local/go/bin
Clone the
gRPC-Goexamples.git clone https://github.com/grpc/grpc-go.gitcd grpc-go/git checkout -b run-observability-example875c97a94dca8093bf01ff2fef490fbdd576373d
Java
After you connect to the VM instance, make sure that you have Java 8 orlater installed.
sudo apt updatesudo apt upgradesudo apt install gitsudo apt-get install -y openjdk-11-jdk-headless
Clone the
grpc-javarepository.export EXAMPLES_VERSION=v1.54.1git clone -b $EXAMPLES_VERSION --single-branch --depth=1 \https://github.com/grpc/grpc-java.git
Create gRPC Google Cloud Observability config file
You need separate gRPC Google Cloud Observability config file to enableMicroservices observability for both server and client. The location ofthis file is exported asGRPC_GCP_OBSERVABILITY_CONFIG_FILE in the futuresteps. Use the following instruction on how to set up the different parametersin the config file.
ExampleGRPC_GCP_OBSERVABILITY_CONFIG_FILE
{"project_id":"your-project-here","cloud_logging":{"client_rpc_events":[{"methods":["google.pubsub.v1.Subscriber/Acknowledge","google.pubsub.v1.Publisher/CreateTopic"],"exclude":true,},{"methods":["google.pubsub.v1.Subscriber/*","google.pubsub.v1.Publisher/*"],"max_metadata_bytes":4096,"max_message_bytes":4096,}],"server_rpc_events":[{"methods":["*"],"max_metadata_bytes":4096,"max_message_bytes":4096}],},"cloud_monitoring":{},"cloud_trace":{"sampling_rate":0.5,}"labels":{"SOURCE_VERSION":"J2e1Cf","SERVICE_NAME":"payment-service-1Cf","DATA_CENTER":"us-west1-a"}}The following sections contain instructions for enabling data collection in yourconfiguration for the individual components. If you used the gRPC example inthis tutorial, you can use this config as is (after updatingyour-project-here) or use this as a template for your application.and anexample showing the configuration information in an environment variable.
Enable metrics
To enable metrics, add thecloud_monitoring object to the configuration andset its value to{}.
For more information about metrics, seeMetrics definitions.
Enable tracing
To enable tracing, do the following:
- Add the
cloud_traceobject to the configuration. - Set the
cloud_trace.sampling_rateto0.5which to randomly traces 50% ofRPCs.
If you plan to enable tracing across services, ensure that the servicessupport the propagation of trace context received from upstream (or startedby itself) to downstream.
For more information about tracing, seeTrace definitions.
Enable logging
To enable logging, do the following:
- Add the
cloud_loggingobject to the configuration. - Add a pattern to either or both of
client_rpc_eventsandserver_rpc_eventsspecifying the set of services or methods for which you want to generatetransport-level event logging and the number of bytes to log for headersand messages.
For more information about logging, seeLog record definitions.
Instrument your applications for the observability plugin
To instrument your applications so that they can use theMicroservices observability plugin, use the following instructions for yourpreferred language.
C++
You can use C++ with Microservices observability as of gRPC C++ v1.54. Theexample repository is in GitHub.
Observability support is only available through the Bazel build system. Addthe target
grpcpp_gcp_observabilityas a dependency.Opting in Microservices observability requires an additionaldependency (an observability modul) and the following code changes toexisting gRPC clients, servers, or both:
#include <grpcpp/ext/gcp_observability.h>intmain(intargc,char**argv){autoobservability=grpc::GcpObservability::Init();assert(observability.ok());…// Observability data flushed when object goes out of scope}Before any gRPC operations, including creating a channel, server, orcredentials, invoke the following:
grpc::GcpObservability::Init();This returns
absl::StatusOr<GcpObservability>which should be saved. Thestatus helps determine whether observability was successfully initialized.The accompanyingGcpObservabilityobject controls the lifetime ofobservability, and automatically closes and flushes observability datawhen it goes out of scope.
Go
- Microservices observability plugins are supported for gRPC Go versions
v1.54.0and later. Theexample repository is in GitHub.
With the Go module, opting in Microservices observability requires anobservability module and the following code:
import"google.golang.org/grpc/gcp/observability"funcmain(){ctx,cancel:=context.WithTimeout(context.Background(),time.Second)defercancel()iferr:=observability.Start(ctx);err!=nil{log.Warning("Unable to start gRPC observability:",err)}deferobservability.End()…}Theobservability.Start call parses the configuration from environmentvariables, creates exporters accordingly, and injects collection logic to clientconnections and servers created after the call. The deferredobservability.Endcall cleans up resources and ensures that buffered data is flushed before theapplication closes.
After the application code is updated, run the following command to update thego.mod file.
gomodtidyJava
To use Microservices observability with Java applications, modify your build toinclude thegrpc-gcp-observability artifact. Use gRPC version 1.54.1 orlater.
In the build snippets in the Gradle and Maven build tool sections,grpcVersion is set to the value 1.54.1.
Theexample repository is in GitHub.
- To successfully instrument your Java applications for Microservicesobservability, add the following code to
main().
...importio.grpc.gcp.observability.GcpObservability;...// Main application class...publicstaticvoidmain(String[]args){...// call GcpObservability.grpcInit() to initialize & get observabilityGcpObservabilityobservability=GcpObservability.grpcInit();...// call close() on the observability instance to shutdown observabilityobservability.close();...}Note that you must callGcpObservability.grpcInit() before any gRPC channelsor servers are created. TheGcpObservability.grpcInit() function reads theMicroservices observability configuration and uses that to set up the globalinterceptors and tracers that are required for the logging, metrics, and tracefeatures in each channel and server created.GcpObservability.grpcInit()is thread safe and must be called exactly once. It returns an instance ofGcpObservability that you must save in order to callclose() later.
GcpObservability.close() de-allocates resources. Any channel or servers createdafterwards don't perform any logging.
GcpObservability implementsjava.lang.AutoCloseable, which is closedautomatically if you usetry-with-resources as follows:
...importio.grpc.gcp.observability.GcpObservability;...// Main application class...publicstaticvoidmain(String[]args){...// call GcpObservability.grpcInit() to initialize & get observabilitytry(GcpObservabilityobservability=GcpObservability.grpcInit()){...}// observability.close() called implicitly...}Use the Gradle build tool
If you are using the Gradle build tool, include the following:
def grpcVersion = '1.54.1'...dependencies {...implementation "io.grpc:grpc-gcp-observability:${grpcVersion}"...}Use the Maven build tool (pom.xml)
If you are using the Maven build tool, include the following:
<properties>...<grpc.version>1.54.1</grpc.version>...</properties>...<dependencies>...<dependency><groupId>io.grpc</groupId><artifactId>grpc-gcp-observability</artifactId><version>${grpc.version}</version></dependency>...</dependencies>Run your application
Follow the instructions in this section only if you used the gRPC example forthe tutorial. You can modify therun command to target your applicationbinary.
Run Server
C++
- Create a SSH session into the VM.
Export environmental variables. Use steps definedabove to create
server_config.json.exportGOOGLE_CLOUD_PROJECT=$PROJECT_IDexportGRPC_GCP_OBSERVABILITY_CONFIG_FILE="$(pwd)/examples/cpp/gcp_observability/helloworld/server_config.json"Run the server application
shellcd grpctools/bazel run examples/cpp/gcp_observability/helloworld:greeter_server
Go
- Create a SSH session into the VM.
Export environmental variables. Use steps definedabove to create
server_config.json.exportGRPC_GCP_OBSERVABILITY_CONFIG_FILE=./server/serverConfig.jsonRun the server application
shellgo run ./server/main.go
Java
- In the examples directory, open the README file and follow theinstructions in the file.
- When the instructions tell you to open another terminal window, issuethis command:
shellgcloud compute ssh --project=$PROJECT_ID grpc-observability-vm
Run Client
C++
- Create another SSH session into the VM.
Export environmental variables. Use steps definedabove to create
client_config.jsonfile.exportGOOGLE_CLOUD_PROJECT=$PROJECT_IDexportGRPC_GCP_OBSERVABILITY_CONFIG_FILE="$(pwd)/examples/cpp/gcp_observability/helloworld/client_config.json"Run the client application
cdgrpctools/bazelrunexamples/cpp/gcp_observability/helloworld:greeter_client
Go
- Create another SSH session into the VM.
- Export environmental variables. Use steps definedabove to create
client_config.jsonfile.shellexport GRPC_GCP_OBSERVABILITY_CONFIG_FILE=./client/clientConfig.json Run the client application
cdgrpc-go/examples/features/observabilitygorun./client/main.go
Java
- In the examples directory, open the README file and follow the instructionsin the file.
- When the instructions tell you to open another terminal window, issue thiscommand:
shellgcloud compute ssh --project=$PROJECT_ID grpc-observability-vm
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.
Delete the project
Delete individual resources
- In the Google Cloud console, go to theVM instances page.
- Select the checkbox for the instance that you want to delete.
- To delete the instance, clickMore actions, clickDelete, and then follow the instructions.
What's next
- For detailed information about configuration data, trace definitions, metricsdefinitions, and log definitions seeMicroservices observability reference.
- For information about viewing logs, traces, and metrics, seeView traces, metrics, and log entries
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-10-27 UTC.