This page describes how you can configure your alerting policy documentationso that notifications provide incident responders with resources andadditional information for incident resolution.

Documentation structure

The documentation of an alerting policy consists of a subject, content,and links. You can configure documentation in the Google Cloud console, theCloud Monitoring API, and the Google Cloud CLI.

Subjects

Thesubject of your documentation appears in the subject ofnotifications for incidents related to your alerting policy.Notification recipients can manage and sort their notifications by subject.

Subject lines are limited to 255 characters. If you don't define a subject inyour documentation, then Cloud Monitoring determines the subject line.Subject lines support plain text andvariables.

Content

Thecontent of your documentation appears in the following notificationtypes:

• Email, in thePolicy Documentation section
• PagerDuty
• Pub/Sub
• Slack
• Webhooks

We recommend configuring your content so that incidentresponders can view remediation steps and incident information innotifications related to your alerting policy. For example, you might configurethe documentation to include a summary of the incident andinformation about relevant resources.

Documentation content supports the following:

• Plain text
• Variables
• Channel-specific controls

• Markdown

Links

You can add links to your documentation so that incident responders canaccess resources such as playbooks, repositories, and Google Cloud dashboardsfrom a notification.

The Cloud Monitoring API lets you define an object that contains the most relevantlinks for responders. While the Google Cloud console doesn't have a fieldspecifically for links, you can add a section for links in your documentationbody.

"links"[{"displayName":"Playbook","url":"https://myownpersonaldomain.com/playbook?name=${resource.type}"}]

### Troubleshooting and Debug ReferencesPlaybook: https://myownpersonaldomain.com/playbook?name=${resource.type}

Markdown in documentation content

You can use Markdown to format your documentation content.Documentation content supports the following subset of Markdown tagging:

• Headers, indicated by initial hash characters.
• Unordered lists, indicated by initial plus, minus, or asterisk characters.
• Ordered lists, indicated by an initial number followed by a period.
• Italic text, indicated by single underscores or asterisks around a phrase.
• Bold text, indicated by double underscores or asterisks around a phrase.
• Links, indicated by[link text](url) syntax. To addlinks to your notification, we recommend that you use theCloud Monitoring API and configure theLink object.

For more information about this tagging, see any Markdown reference,for example,Markdown guide.

Variables in documentation

To customize the text in your documentation, you can use variablesof the form${varname}. When the documentation is sent witha notification, the string${varname} is replaced with a value drawn from thecorresponding Google Cloud resource, as described in the following table.

¹ For example,${resource.label.zone} is replaced withthe value of thezone label. The values of these variables are subject togrouping; seenull values for more information.
² To retrieve the value of theproject_id label on amonitored resource in the alerting policy, use${resource.project}.
³ You can't access user-defined resource metadata labels byusingresource.label.KEY. Usemetadata.user_label.KEYinstead.

Usage notes

• Only the variables in the table are supported. You can't combine them intomore complex expressions, like${varname1 + varname2}.
• To include the literal string${ in your documentation, escape the$symbol with a second$ symbol, and$${ renders as${ in yourdocumentation.
• These variables are replaced by their values only in notifications sentthrough notification channels. In the Google Cloud console, when thedocumentation is shown, you see the variables, not the values. Examplesin the console include the descriptions of incidents and the preview ofthe documentation when creating an alerting policy.
• Verify that the aggregation settings of the condition don't eliminate thelabel. If the label is eliminated, then the value of the label in thenotification isnull. For more information,seeVariable for a metric label is null.

• Theresource.label.KEY andmetric.label.KEY variables can havenull values if your alerting policy uses cross-series aggregation(reduction), for example, calculating the SUM across each of thetime-series that match a filter. When using cross-series aggregation,any labelsnot used in grouping are dropped and as a result they render asnull when the variable is replaced with its value. All labelsare retained when there is no cross-series aggregation.For more information,seeVariable for a metric label is null.

• Values formetadata.* variables are available only ifthe labels are explicitly included in a condition's filter or grouping forcross-series aggregation. That is, you must refer to the metadata label ineither filter or grouping for it to have a value for the template.

Variable resolution

Variables in documentation templates are resolved only in the notificationssent by using the following notification channels:

• Email
• Google Chat
• Slack
• Pub/Sub, JSON schema version 1.2
• Webhooks, JSON schema version 1.2
• PagerDuty, JSON schema version 1.2

<@backendoncall>Incidentcreatedbasedonpolicy${policy.display_name}

The following example shows Google Cloud console and Cloud Monitoring API versions oftemplate documentation for a CPU utilization alerting policy.These examples use an email for thenotification channel type.The documentation templates include several variables to summarize theincident and to reference the alerting policy and condition RESTresources.

"documentation":{"content":"### CPU utilization exceeded\n\n### Summary\n\nThe ${metric.display_name} of the ${resource.type} ${resource.label.instance_id} in the project ${resource.project} has exceeded 5% for over 60 seconds.\n\n#### Additional resource information\n\nCondition resource name: ${condition.name}  \nAlerting policy resource name: ${policy.name}","mimeType":"text/markdown","subject":"Alert: ${metric.display_name} exceeded","links":[{"displayName":"Playbook","url":"https://myownpersonaldomain.com/playbook?name=${resource.type}"},{"displayName":"Repository with debug scripts","url":"https://altostrat.com"},{"displayName":"Google Cloud dashboard","url":"https://example.com"}]}

### CPU utilization exceeded#### SummaryThe ${metric.display_name} of the ${resource.type}${resource.label.instance_id} in the project ${resource.project} hasexceeded 5% for over 60 seconds.#### Additional resource informationCondition resource name: ${condition.name}  Alerting policy resource name: ${policy.name}  #### Troubleshooting and Debug ReferencesPlaybook: https://myownpersonaldomain.com/playbook?name=${resource.type}  Repository with debug scripts: https://altostrat.com  ${resource.type} dashboard: https://example.com