Learn how to collect and view latency data from your applications:

1. Create a Google Kubernetes Engine (GKE) clusterby using the Google Cloud CLI.

2. Download and deploy a sample application to your cluster.

3. Create a trace by sending an HTTP request to the sample application.

4. View the latency information of the trace you created.

5. Clean up.

To follow step-by-step guidance for this task directly in the Google Cloud console, clickGuide me:

Guide me

Before you begin

1. Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, seeDevelop applications in a constrained Google Cloud environment.

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

If you're using an existing project for this guide,verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Google Kubernetes Engine and Cloud Trace APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

If you're using an existing project for this guide,verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Google Kubernetes Engine and Cloud Trace APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

Required roles

To get the permissions that you need to create a Google Kubernetes Engine cluster and to view trace spans, ask your administrator to grant you the following IAM roles on your project:

• Kubernetes Engine Admin (roles/container.admin)
• Cloud Trace User (roles/cloudtrace.user)
• Service Account User (roles/iam.serviceAccountUser)
• Service Usage Admin (roles/serviceusage.serviceUsageAdmin)

For more information about granting roles, seeManage access to projects, folders, and organizations.

You might also be able to get the required permissions throughcustom roles or otherpredefined roles.

Create a GKE cluster

This guide creates a standard GKE cluster. If you useAutopilot mode for Google Kubernetes Engine or if you enableWorkload Identity Federation for GKE, then you mustconfigure your application to use Workload Identity Federation for GKE.

1. In the toolbar, clickterminalActivate Cloud Shell, and then perform the following stepsin the Cloud Shell.

2. Create a cluster:

   gcloudcontainerclusterscreatecloud-trace-demo--zoneus-central1-c

   The previous command, which takes several minutes to complete,creates a standard cluster with the namecloud-trace-demo in the zoneus-central1-c.

3. Configurekubectl to automatically refresh its credentials to usethe same identity as the Google Cloud CLI:

   gcloudcontainerclustersget-credentialscloud-trace-demo--zoneus-central1-c

4. Verify access to your cluster:

   kubectlgetnodes

   A sample output of this command is:

   NAME                                              STATUS   ROLES    AGE   VERSIONgke-cloud-trace-demo-default-pool-063c0416-113s   Ready    <none>   78s   v1.22.12-gke.2300gke-cloud-trace-demo-default-pool-063c0416-1n27   Ready    <none>   79s   v1.22.12-gke.2300gke-cloud-trace-demo-default-pool-063c0416-frkd   Ready    <none>   78s   v1.22.12-gke.2300

Download and deploy an application

Download and deploy a Python application, whichuses the Flask framework and the OpenTelemetry package. The applicationis described in theAbout the app section of this page.

In the Cloud Shell, do the following:

1. Clone a Python app from GitHub:

   gitclonehttps://github.com/GoogleCloudPlatform/python-docs-samples.git

2. Run the following command to deploy thesample application:

   cdpython-docs-samples/trace/cloud-trace-demo-app-opentelemetry &&./setup.sh

   The scriptsetup.sh takes several minutes to complete.

   The script configures three services using a prebuilt image and then waitsfor all resources to be provisioned.The workloads are namedcloud-trace-demo-a,cloud-trace-demo-b, andcloud-trace-demo-c.

   A sample output of this command is:

   deployment.apps/cloud-trace-demo-a is createdservice/cloud-trace-demo-a is createddeployment.apps/cloud-trace-demo-b is createdservice/cloud-trace-demo-b is createddeployment.apps/cloud-trace-demo-c is createdservice/cloud-trace-demo-c is createdWait for load balancer initialization complete......Completed.

Create trace data

Atrace describes the time it takes an application to complete asingle operation.

To create a trace, in the Cloud Shell, run the following command:

curl$(kubectlgetsvc-o=jsonpath='{.items[?(@.metadata.name=="cloud-trace-demo-a")].status.loadBalancer.ingress[0].ip}')

The response of the previous command looks like the following:

Hello, I am service AAnd I am service BHello, I am service C

You can execute thecurl command multiple times to generate multipletraces.

View latency data

1. In the Google Cloud console, go to theTrace explorer page:

   Go toTrace explorer

   You can also find this page by using the search bar.

   The following screenshot shows the result of running thecurl commandseveral times. The graph displays aggregated latency data forspans, whichdescribe trace sub-operations.

   

   Note: If you don't see any trace data after waiting a few minutes, then theremight be a permission problem. To resolve this problem, check to see whetherthe service account with the extension@developer.gserviceaccount.com has the Editorrole. If not, thengrant the service accounttheCloud Trace Agentrole, and run thecurl command.

2. To view a trace in detail, select a span in the graph or a row in the table.

   The trace details dialog opens, as shown in the following screenshot:

   

   A Gantt chart displays information about the selected trace. The firstrow in the Gantt chart is for the trace, and each following row is a spanwithin the trace.

3. To view information detailed information about a span, in the Gantt chart,select the span.

About the application

The sample application used in this quickstart is available in a GitHubrepository. This repository contains information on how to use the applicationin environments other than the Cloud Shell.The sample application is written in Python, uses the Flask framework andOpenTelemetry packages, and executes on a GKE cluster.

This guide creates a standard GKE cluster. If you useAutopilot mode for Google Kubernetes Engine or if you enableWorkload Identity Federation for GKE, then you mustconfigure your application to use Workload Identity Federation for GKE.

Instrumentation

The fileapp.py in theGitHub repository,contains the instrumentation necessary to capture and send tracedata to your Google Cloud project:

• The application imports several OpenTelemetry packages:

  fromopentelemetryimporttracefromopentelemetry.exporter.cloud_traceimportCloudTraceSpanExporterfromopentelemetry.instrumentation.flaskimportFlaskInstrumentorfromopentelemetry.instrumentation.requestsimportRequestsInstrumentorfromopentelemetry.propagateimportset_global_textmapfromopentelemetry.propagators.cloud_trace_propagatorimportCloudTraceFormatPropagatorfromopentelemetry.sdk.traceimportTracerProviderfromopentelemetry.sdk.trace.exportimportBatchSpanProcessor

• The application instruments web requests with trace context and automaticallytraces Flask handlers and requests to other services:

  app=flask.Flask(__name__)FlaskInstrumentor().instrument_app(app)RequestsInstrumentor().instrument()

• The application configures the Cloud Trace exporter as a trace provider,which propagates trace context in the Cloud Trace format:

  defconfigure_exporter(exporter):"""Configures OpenTelemetry context propagation to use Cloud Trace context    Args:        exporter: exporter instance to be configured in the OpenTelemetry tracer provider    """set_global_textmap(CloudTraceFormatPropagator())tracer_provider=TracerProvider()tracer_provider.add_span_processor(BatchSpanProcessor(exporter))trace.set_tracer_provider(tracer_provider)configure_exporter(CloudTraceSpanExporter())tracer=trace.get_tracer(__name__)

• The following code snippet shows how to send requests in Python.OpenTelemetry implicitly propagates the trace context for you with youroutgoing requests:

  ifendpointisnotNoneandendpoint!="":data={"body":keyword}response=requests.get(endpoint,params=data,)returnkeyword+"\n"+response.textelse:returnkeyword,200

How the application works

For clarity, in this section,cloud-trace-demo is omitted from the servicenames. For example, the servicecloud-trace-demo-c is referenced asc.

This application creates three services nameda,b, andc. Servicea isconfigured to call serviceb, serviceb is configured to call servicec.For details on the configuration of the services, see the YAML files in theGitHub repository.

When you issued a HTTP request to servicea in this quickstart,you used the followingcurl command:

curl $(kubectl get svc -o=jsonpath='{.items[?(@.metadata.name=="cloud-trace-demo-a")].status.loadBalancer.ingress[0].ip}')

Thecurl command works as follows:

1. kubectl fetches the IP address of the service namedcloud-trace-demo-a.
2. Thecurl command then sends the HTTP request to servicea.
3. Servicea receives the HTTP request and sends a request to serviceb.
4. Serviceb receives the HTTP request and sends a request to servicec.
5. Servicec receives the HTTP request from serviceb and returns thestringHello, I am service C to serviceb.
6. Serviceb receives the response from servicec, appends it to thestringAnd I am service B, and returns the result to servicea.
7. Servicea receives the response from serviceb and appends it to thestringHello, I am service A.
8. The response from servicea is printed in the Cloud Shell.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

If you created a new project and you no longer need the project, thendelete the project.

If you used an existing project, then do the following:

1. To delete your cluster, in the Cloud Shell, run the following command:

   gcloud container clusters delete cloud-trace-demo --zone us-central1-c

What's next

• For information on languages and platforms supported, seeCloud Trace overview.

• For details on how to instrument your applications, see:

  • Instrument for Cloud Trace.
  • OpenTelemetry.
  • Data model.

• For more information on theTrace Explorer window, seeFind and view traces.

• To learn more about managing GKE clusters, seekubectl.