This page provides a conceptual overview of log-based metrics. These metricscan help you observe trends and patterns in a large volume of log entries.

About log-based metrics

Log-based metrics areCloud Monitoring metricsthat are derived from the content of log entries. For example, you can use alog-based metric to count the number of log entries that contain a particularmessage or to extract latency information recorded in log entries.You can display log-based metrics in Cloud Monitoring charts, and alertingpolicies can monitor these metrics.

Log-based metrics can apply at the project level or log bucket level.Bucket-scoped log-based metrics are useful when you use aggregated sinksto route log entries to a log bucket, and when you route log entries fromone project to a log bucket in another project. For more information, seethe section of this page titledSources of log-based metrics.

How time-series values for log-based metrics are determined

This section briefly describes how the values in a time series for acounter-type log-based metric are determined. It also describes why atime series for a log-based metric might have gaps.The process is similar for distribution-valued log-based metrics.

For counter-type log-based metrics,Logging does the following:

1. It creates one or more data streams and for each stream, it manages a set ofin-memory counters.

   There is one stream for each combination ofunique values for the set of metric and resource labels.Each in-memory counter is for a different time interval,but each counter records the number of log entries that match the filter andthat have a timestamp which corresponds to the interval.

2. Every minute, for each stream, the Logging backend uses thevalues of the in-memory counters to determine how to update a time seriesthat is managed by Cloud Monitoring.

   If there are five streams, then there are five time series managed byMonitoring. You can chart these time series. You can alsocreate alerting policies to monitor these time series.

   Note: A time series for a log-based metric can include values of zero andcan have data gaps. If you plan to create an alerting policy to monitora log-based metric, then we recommend that you configurehow the policyhandles missing data.

Example: How zero values can cause data gaps

Consider the following time series:

Example time series: (T1, -), (T2, -), (T3, 0), (T4, 1), (T5, 0), (T6, -)

In the previous expression,(T1, -) means that no value was writtenfor timeT1. This corresponds to a gap in the values in the time series.In contrast(T3, 0) means that the value of the time seriesat timeT3 is zero.

For each stream, the backend system for log-based metrics uses the followingrules to determine what value to write to the time series stored byMonitoring:

• When the aggregated count for an interval is non-zero, that value iswritten to the time series.

  In the example time series, only the intervalT4 has anon-zero value. The value of1 indicates that one log entry with atimestamp that corresponds to theT4 intervalmatched the filter of the log-based metric.

• When the aggregated count for an interval is zero, a value of zero is writtento the time series when an adjacent interval has a non-zero value.

  In the example time series, at timeT3, the aggregated count is zero.However, a value of zero is written because at timeT4 the aggregated countis non-zero. Similarly, at timeT5, the aggregated count is zero. A valueof zero is written to the time series because the value for the previousinterval was non-zero.

• Otherwise, the time series isn't updated and there is a data gap for theinterval.

  In the example time series, there is no value for intervalsT1,T2, andT6. In these intervals, the aggregated counts were zero and the counts forthe adjacent intervals were also zero.

Handling of late-arriving and future log entries

When log entries arrive with a timestamp for an interval thathas been processed, the Logging backendgenerates historical data. The historical data is used to update thepreviously-written counts.

When log entries arrive with a timestamp that is in the future, theLogging backend accumulates these counts. In the time series,these log entries are counted when their timestamp corresponds to thecurrent interval.

Sources of log-based metrics

You can use the metrics defined by Cloud Logging to collect general usageinformation, and you can define your own log-based metric to capture informationspecific to your application or business.

Log-based metrics can apply within a single Google Cloud project or within a logbucket. You can't createlog-based metrics for other Google Cloud resources such asCloud Billing accounts or organizations.

For information about the differences between project-based log-based metricsand bucket-based log-based metrics, seeBucket-scoped log-based metrics.

System-defined metrics

Logging provides a set of metrics for usage values such as thenumber of log entries stored in log buckets in your project, orthe number of bytes you've exported.For a complete list of system-defined metrics, seeGoogle Cloud metrics: logging.

System-defined log-based metrics apply at the project level.

The Log Router counts a log entry when all of the following are true:

• The log entry passes through the log sinks of the project where the log-based metric is defined.

• The log entry is stored in a log bucket. The log bucket can be in any project.

  For example, suppose projectA has a log sink whose destination is projectB. Also assume that the log sinks in the projectB route the log entries to a log bucket. In this scenario, the log entries routed from projectA to projectB contribute to projectA's system-defined log-based metrics. These log entries also contribute to projectB's system-defined log-based metrics.

User-defined metrics

You can create user-defined log-based metrics totrack other metrics that are important for your project. Forexample, you might create a log-based metric to count the number of logentries that match a given filter.

By default, user-defined log-based metrics are calculated from all logsreceived by the Logging API for the Google Cloud project,regardless of anyinclusion filters orexclusion filters that might apply to theGoogle Cloud project.

You can also create user-defined log-based metricsfor a specific log bucket in a Google Cloud project. Bucket-scopedlog-based metrics are calculated from all logs destined for the bucket,regardless of where they originated. For more information seeLog-based metrics on log buckets.

The Log Router counts a log entry when all of the following are true:

• Billing is enabled on the project where the log-based metric is defined.
• For bucket-scoped metrics, the log entry is stored in the log bucket where the log-based metric is defined.
• For project-scoped metrics, the log entry passes through the log sinks of the project where the log-based metric is defined.

Data types for log-based metrics

Log-based metrics can extract data from logs to create metrics of thefollowing types:

• Counter: these metrics count the number of log entries that match aspecified filter within a specific period. Use counters when you want to keeptrack of the number of times a value or string appears in your logs.
• Distribution: these metrics also count values, but they collect thecounts into ranges of values (histogram buckets). Use distributions when youwant to extract values like latencies.
• Boolean: these metrics capture whether or not a log entry matches a specifiedfilter.

User-defined log-based metrics can be of the counter or distribution metrictypes. Most system-defined log-based metrics are counters, but some are ofthe boolean type. The characteristics of counters and distributions aredescribed in more detail in subsequent sections.

The data for a user-defined log-based metric comes only from log entriesreceivedafter the metric is created. A metric isn't retroactivelypopulated with data from log entries that are already in Logging.

System log-based metrics are calculated from included logs only. User-definedlog-based metrics are calculated from both included and excluded logs.

The following sections describe the characteristics of counter-type anddistribution-type metrics.

Counter metrics

Counter metrics count the number of log entries matching a given filter. Forexample, you can do the following:

• Count the log entries that contain a certain specific error message.

• Count the number of times each user invokes an operation, by looking for logmessages that match this pattern:

  ... userUSERNAME calledOPERATION ...

  By extractingUSERNAME andOPERATION and using them asvalues for twolabels, you can later ask, "How many times didsally call theupdate operation?", "How many people called thereadoperation?", "How many times didgeorge call an operation?", and so on.

For more information, seeConfigure counter metrics.

Distribution metrics

Distribution metrics accumulate numeric data from log entries matching afilter. The metrics contain a time series of distribution objects, each ofwhich contains the following:

• A count of the number of values in the distribution.
• The mean of the values.
• The sum of squared deviations:Sum_i=1..n(x_i–mean)²
• A set of histogram buckets with the count of values in each bucket. You canuse the default bucket layout or choose your own.

A common use for distribution metrics is to track latencies. As each log entryis received, a latency value is extracted from somewhere in the log entry and isadded to the distribution. At regular intervals, the accumulated distribution iswritten to Cloud Monitoring.

For information about distributions, including their format within a time seriesand how they are visualized, seeCharting distribution metrics.

For information about creating distribution log-based metrics, seeConfigure distribution metrics.

Resource types

We recommend that your log-based metrics specify a single resource type.This specification ensures that the time series generated for yourlog-based metric matches the data model expected by Cloud Monitoring.It also simplifies the process of creating charts and alerting policies, asthere is no ambiguity as to which time-series data is charted or monitored.

Consider a log-based metric with the following filter:

severity>="ERROR"resource.type="gce_instance"

Cloud Monitoring recognizes that this data is for a Compute EngineVM instance. Therefore, when you create a chart for a VM instance, yourlog-based metric is listed as an option in the menus. When youcreate a chart for a different resource type, yourlog-based metric isn't listed as an option.

If you create a log-based metric without specifying a resource type, thenMonitoring lists your log-based metric as an option for multipleresource types, even though you don't count entries from those resources.

Labels

Log-based metrics can have labels, which allow multiple time seriesto be collected for the metric. Values for the labels are extracted from fieldsin the matching log entries. Logging records separate time seriesfor each combination of label values.

Most log-based metrics provide a defaultlog label.This label holds the value of theLOG_ID portion of thelogName field in the log entry, for example,cloudresourcemanager.googleapis.com%2Factivity.

System log-based metrics have predefinedlabels that can't be changed.

User-defined log-based metrics can include labels that you define. Typically,these labels extract values from the log entries that contribute to thelog-based metric.

For example, you can use labels in the log-based metric to record the folderor organization from which a log entry originated, even though bucket-scopedlog-based metrics can't be created for folders or organizations. For moreinformation, seeLog-based metric labels.

Charts and alerting policies in Cloud Monitoring

You can use both system and user-defined log-based metrics inCloud Monitoring to create charts and alerting policies. For these chartsand alerting policies, you can use labels to filter or combine time series.For example, you can set a filter to show only certain time series. For moreinformation, seeList and chart log-based metrics andAlert on log-based metrics.

In Cloud Monitoring, log-based metrics use the following naming patterns:

• System:logging.googleapis.com/SYSTEM_METRIC_NAME
• User-defined:logging.googleapis.com/user/USER_METRIC_NAME

Note that user-defined log-based metrics include the stringuser.

Visibility to Monitoring metrics scopes

Log-based metrics are ingested by Cloud Monitoring, and the visibilityof metric data to a Google Cloud project is determined by a metrics scope.Ametrics scope is a list of projects that are monitored by the projectthat hosts the metrics scope; the hosting project is called ascoping project.

By default, each project hosts a metrics scope thatincludes only itself, so a project is a scoping project for itself.Therefore, your metrics, including log-based metrics, are visible only to yourGoogle Cloud project.

You can also create a multi-project metrics scope for thescoping project. With a multi-project metrics scope, thescoping project can see the metrics from all the projectsin the metrics scope. What is visible to the individual projects in amulti-project metrics scope is determined by the metrics scopehosted by each of those projects. The fact that two projects are in amulti-project metrics scope does not mean that each project has access tothe metric or configuration data in the other project.

A single project can also appear in multiple metrics scopes.The metrics from such a project are visible tothe scoping projects of each of those metrics scopes.

Metrics, including log-based metrics, are defined within a specific project.When that project appears in multiple metrics scopes,the metrics are visible to projects other than the one in which they aredefined. For information about how metrics scopes and bucket-scoped log-basedmetrics interact, seeProjects and metrics scopes.

For more information about metrics scopes, including multi-projectmetrics scopes, and about scoping projects, see the following:

• Metrics scopes
• View metrics for multiple projects

Limitations

You can't create log-based metrics that count the number of log entriesthat contain an error group. You also can't extract the ID of anerror group into a label attached to a log-based metric.

Pricing

Alluser-defined log-based metrics are a class ofCloud Monitoring custom metrics and are chargeable. For pricing information,seeGoogle Cloud Observability pricing.

Quota

For information about the quotas and limits associated with user-definedlog-based metrics, seeQuotas and limits.

Troubleshooting

If you encounter issues when using log-based metrics, seeTroubleshoot log-based metrics.