Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

This document explains how to create an alerting policy to monitor the resultsof a query you run in Log Analytics. These queries are written in SQL and theymust query alog view. The alerting policynotifies you when the query resultsatisfies conditions that you specify. For example, you couldconfigure an alerting policy so that you're notified when at least 25% of thelog entries in a certain period have a severity ofERROR.

Alerting policies that you create from theLog Analytics page run ona BigQuery engine. Therefore, the data being queried must beaccessible through alinked BigQuery dataset.For this reason, these SQL queries can only query log views. They can't queryanalytics views.

For general information about Log Analytics, seeQuery and analyze logs with Log Analytics.

Note: If your data is managed through anAssured Workloads environment,then this feature might be impacted or restricted. For information, seeRestrictions and limitations in Assured Workloads.

How alerting policies work

An alerting policy describes the circumstances under which you want to bealerted and how you want to be notified about an incident.There are three different approaches that you can use to get notifiedwhen content or patterns appear in your log data:

• To scan individual log entries for a specific phrase, create alog-based alerting policy. Use these alerting policies when youwant to be notified about things like security-related events.

• To monitor events in your log entry data, you can create alog-based metric and then create an alerting policy to monitor themetric. These types of alerting policies are effective when you want tomonitor trends in log entry data over time. However, they aren't as effectiveif you expect only a few events.

• To monitor aggregate analysis of your log entry data, combine Log Analyticswith alerting policies. In this scenario, you upgrade a log bucket to useLog Analytics and create alinked BigQuery dataset for that log bucket.Next, you use Log Analytics, which supports SQL queries,to query a log view on the log bucket. Finally, youcreate the alerting policy to monitor the SQL-query results.This type of alerting policy is called aSQL-based alerting policy.

SQL-based alerting policies are most effective for evaluatingexact values over multiple log entries. If you want to evaluate individuallog entries, then create alog-based alerting policy.

The remainder of this document describes how to use SQL-based alerting policies.

Alerting policy components

A SQL-based alerting policy contains a condition and a schedule:

• Thecondition contains thequery, which is a SQL query that queriesa log view. The conditionalso defines the circumstances under which the query resultcauses Monitoring to create an incident.

• Theschedule defines how frequently the alerting policy runs its query.The schedule also defines the size of thelookback window, which is a filterthat selects only those log entries that have beenreceived since the previous time the query was evaluated. For example, if youset the schedule to 60 minutes, then the query is runevery 60 minutes using a lookback window that selects the most-recent 60minutes of log entries.

Alerting policies also contain a list ofnotification channels.When the condition of the alerting policy is met,Cloud Monitoring creates an incident and then sendsnotifications about the incident through these channels.Anincident is a record of the data that caused the condition to bemet along with other relevant information. Thisinformation can help you troubleshoot the issues that caused the incident.You can view the incident by using the Google Cloud console.

Evaluation types for SQL-based alerting policies

Conditions that monitor a SQL query result support two types of evaluation:

• Row count threshold: The condition is met when the number of rows in thequery result is greater than, equal to, or less than athreshold value.

  For example, suppose you want to get notified when more than 50log entries in the lookback window have a severity greater than 200.You create a query that reports log entries whose severityis greater than 200. You then configure a condition, select theRow count threshold, and set the threshold to 50.

• Boolean: The condition is met when a specific boolean column in the queryresult table contains any row with a value oftrue.

  For example, suppose you want to get notified when more than 25% of thelog entries in the lookback windowhave a severity ofERROR. You create a query that computes the percentageof log entries whose severity level isERROR. The query results writestrue to thenotify column when that percentage exceeds 25%. Next,you create a condition, set the type toBoolean, and configure thecondition to monitor thenotify column.

Alerting policies that monitor a SQL query result must have only one condition.

Alerting policies and BigQuery

When an alerting policy runs a SQL query, that query is run using reservedBigQuery slots in the Google Cloud project where thealerting policy is defined. For more information, seeWork with slot reservations.

For an alerting policy to use reserved BigQuery slots to querya log view, the log bucket that hosts the log view must be configured to have alinked BigQuery dataset. Linked datasets letBigQuery read the data in the log bucket, and they let youperform BigQuery functions on the datareturned by your SQL query.

Evaluated log entries

For a log entry to be evaluated by the SQL query of an alerting policy,both of the following must be true:

• The log entry'sreceived timestamp, which records when the log entry wasreceived by Cloud Logging, must be within the lookback window of thealerting policy.
• The log entry'stimestamp, which records when the log entry wasgenerated, must be within 15 minutes of the lookback window.

For example, your SQL-based alerting policy has a 60-minute lookback window.Log Analytics runs the SQL query of the alerting policy at 1:30 PM. To beincluded in the query, a log entry must match both of the following criteria:

• The received timestamp must be between 12:30 PM and 1:30 PM.
• The timestamp must be between 12:15 PM and 1:45 PM.

When you run a query from the Log Analytics interface, all log entries in theselected time range are evaluated based on the log entry's timestamp.

Lookback window and incident propagation time

When an alerting policy is scheduled to evaluate its condition, Log Analyticsdelays the SQL query execution by five minutes to provide time forCloud Logging to index the log entries received during the lookback window.For example, if the alerting policy uses a lookback window that ends at 2:00 PM,then Log Analytics doesn't execute the SQL query until 2:05 PM.

If the alert condition is met after the query is executed, then it can take upto two additional minutes for the incident to propagate through thesystem.

Query failures

Queries issued by SQL-based alerting policies can fail for various reasons,including the following:

• The Monitoring Service Account no longer exists or it no longer has thenecessary permissions to read the log data that is being queried.

• The query execution time exceeds fives minutes.

• An internal error occurs.

A failed query generates a log entry containing the alert policy ID and theerror status. You can use alog-based alerting policy to create analert when an error is logged.

Before you begin

This section assumes that you haveupgraded your log bucket to use Log Analytics andthat you can query and view your log data by using the Log Analytics page. It alsoassumes that you have alreadycreated a linked BigQuery datasetfor your log bucket.

Before you create a SQL-based alerting policy, complete the following steps:

1. To get the permissions that you need to create and manage SQL-based alerting policies, ask your administrator to grant you the following IAM roles:

   • Monitoring Editor (roles/monitoring.editor) on your project
   • Logging SqlAlertWriter (roles/logging.sqlAlertWriter) on your project

2. Verify that theMonitoring Service Account exists and that it has the following roles:

   1. Monitoring Service Agent (roles/monitoring.notificationServiceAgent) on your project.
   2. BigQuery Data Viewer (roles/bigquery.dataViewer) on your linked dataset.

   If the Monitoring Service Account doesn't exist, then seeTroubleshoot: No Monitoring Service Account.

3. To enable your SQL-based alerting policies to run on reserved BigQuery slots, do the following:
   1. Configure reserved BigQuery slots.
   2. Assign reserved slots to your Google Cloud project.
4. Configure the notification channels that you want to use to receive any notifications for incidents. For redundancy purposes, we recommend that you create multiple types of notification channels. For more information, seeCreate and manage notification channels.

Create a SQL-based alerting policy

To create a SQL-based alerting policy, do the following:

Limitations

• You can have one condition per SQL-based alerting policy.

• SQL-based alerting policies can't query ananalytics view.

• Queries issued by SQL-based alerting policies fail when their execution timeexceeds five minutes.

• There is a delay ofup to seven minutes, plus thequery execution time, between when a query is scheduled and when anincident is created.

For a full list of limits associated with alerting policies, seeMonitoring limits.

Pricing

For information about pricing, see the following documents:

• BigQuery pricing
• The Cloud Logging and Cloud Monitoring sections of theGoogle Cloud Observability pricing page.

What's next

• For information about creating charts from your Log Analyticsdata, seeChart query results with Log Analytics.

• For information about managing incidents for these alerting policies,seeIncidents for SQL-based alerting policies.