This page explains some of the behaviors that you might see when using metricscollected by your Google Cloud projects.

Chart errors

This section describes errors that you might see when you configure a chart.You can configure charts by using Metrics Explorer or by adding themto a dashboard.

Gaps in chart data

Charts occasionally exhibit gaps in the data. For example, the followingscreenshot illustrates this behavior for the metricloadbalancing.googleapis.com/https/total_latencies:

Gaps in the data can occur when data is written very infrequently.Cloud Monitoring collects metrics by sampling, and with very sparse data,there might not be any data available when a sample is collected. For example,gaps in load-balancer metrics might occur when the request count is0.05 queries per second (QPS) or less. When the query rate increases, the gapsdisappear.

Alignment period changes when the time range is modified

You add a chart to a dashboard. When you change the time range for the chart,sometimes the alignment period changes. For example, if you change thetime range from one hour to two hours, the alignment period is listed as1 minute interval. However, if you change the time range from one hour toone day, then the alignment period is listed as5 minute interval.

This behavior is expected.

Charts are limited in thenumber of data points that they can display. Therefore, when the number ofdata points available for the selected time range is larger than thedisplay limit, Monitoring automatically modifies the alignmentsettings to reduce the number of data points to display.

Chart doesn't display all time series

You configure a chart, but it doesn't display all time series.

For performance reasons, charts are configured to display at most50 time series.

When you configure a chart and more than 50 time seriesare available, then the chart limits the number of displayed time seriesand adds to the toolbar an icon with a red dot. The tooltip for the icondisplays the following message:

To improve performance, we've limited the time series displayed in this chart.

To display all time series, expand the tooltip andselect the button labeledShow All Time Series.

Chart doesn't display any data

You configure a chart but the chart displays an error message instead of data:

• When there isn't data available for the time range specified for your chart,the following message is shown:

  No data is available for the selected time frame.

  To resolve the problem, try the following:

  • Increase the time range for the chart. Some metric types, such as thosethat report quota usage, provide only one sample per day.

  • Change the selected metric type for your chart. It is possible thatthe metric type specified by your chart is inactive. That is, that thereisn't data for that metric type.

• When the metric type or resource type specified by the query isn'trecognized, the following message is shown:

  An error occurred requested data. One or more resources could not be found.

  To resolve this problem, try the following:

  • Verify that the metric type and resource type specified in thequery are defined. You can use the menu-driven interface for charts tosearch for the metric type.

  • Verify the spelling of the metric type and resource type in thequery.

Chart displays a warning about sensitive metadata

You open a custom dashboard and expect all widgets to display data.However, one or more widgets that contain SQL queries display messagessimilar to the following:

This SQL query is configured to run on the BigQuery engine using [PROJECT_ID],and references resources from [RESOURCE, ...]. Sensitive metadata,including table contents, may be written to [PROJECT_ID].

The warning message indicates that the query crosses ownership boundariesand it provides you an opportunity to verify the query before it is executed.Ownership boundaries are determined by several factors, one of which is thehierarchy of the resources the query uses.

The system performs boundary checks each time a dashboard is opened. You can'tprevent or dismiss these warnings.

Google Cloud provides some features that can help prevent data exfiltrationthrough phishing, insider attacks, or external entities. To learn more, seeData security with Log Analytics.

Chart with multiple metrics displays an error message

You are configuring a chart to display multiple metrics and the chart displaysthe following error message:

    The units for queries using the same Y-axis must be the same for the chart    to render correctly.

Charts can display multiple metric types, and you can map a metric type toeither the left Y-axis or the right Y-axis. The units of all metric typesmapped to the left Y-axis must be the same; otherwise, the previous errormessage is shown. Similarly, the units of allmetric types mapped to the right Y-axis must be the same or the previous errormessage is shown.

To resolve this error, do the following:

1. If your chart contains metrics with more than two unit types, thenremove metrics until you have only two unit types.

2. Assign all metrics with one unit type to the left Y-axisand all metrics with the other unit type to the right Y-axis.

Chart with multiple metrics displays unexpected data

A chart is configured to display multiple metrics, and filterswere added to show only specific time series. However,the chart shows time series that don't satisfy the filters.

To resolve this situation, ensure that you apply filters to each metricthat is charted.

For example, the following screenshot shows the configurationpane of Metrics Explorer for a chart that displays the number of bytesread, and written, by virtual machine (VM) instances:

In the screenshot, there is one heading for each metric typethat is displayed by the chart.

For the example, if you want the chart to display time series only forVMs that are in theus-central1-a zone, then add the filterzone = us-central1-a to each query.

Unable to select filter values for PromQL-defined charts

You create charts on a custom dashboard and configure those charts withPromQL. You then createpermanent dashboard filters.When you view the dashboard, you attempt to change the default value ofthe filter, but the menu of label values isn't displayed or the menu oflabel values isn't correct:

To resolve this issue, enter the filter value, and then selectApply.For example, if the filter key iszone, then you might enterus-central1-a.

Dashboard errors

This section describes errors that you might see when you create oredit dashboards.

Dashboard create fails

You create a dashboard by using a JSON or YAML description of the dashboard,and that description includes aname field.You change the display name and try to create a second dashboard, but you getan error. For example, if you use the Google Cloud CLI, then the you get thefollowing error:

ERROR: (gcloud.alpha.monitoring.dashboards.create) Resource in projects [xxxxxxx] is the subject of a conflict: Requested entity already exists.

The cause of the failure is that the dashboard ID, which identfies thisdashboard in the value of thename field, already exists. Therefore, thedashboard can't be created.

If you specify thename field when you create a dashboard, then the valuethat you specify must be globally unique. Thename field has the followingsyntax:

"name": "projects/PROJECT_ID_OR_NUMBER/dashboards/DASHBOARD_ID"

The display name doesn't have to be unique.

Dashboard isn't displaying all events

You configure adashboard to display events;however, only recent events are displayed. Older events aren't shown.

To resolve this problem, do one of the following:

• Use a shorter time range. For example, display data for the most recenthour instead of for the most recent day or select a custom time range.

• Add filters.

For performance reasons, dashboards are limited in the number of events thatthey can display. When these limits are reached, the dashboardsdisplay only the most recent events.

Terraform-created dashboard is rescaled

You created a dashboard by using Terraform and then edited the dashboardby using the Google Cloud console. You expected that the dashboard would beunchanged except for your edits. Instead, the dashboard is rescaled.For example, your dashboard had 12 columns and now has 48 columns.

Don't edit dashboards created by using Terraform with theGoogle Cloud console. When the dashboard is edited, the size is scaled bya factor of four.

Alerting policy widget doesn't display any data

A dashboard contains a chart for an alerting policy, but the chart doesn'tdisplay any data.

To resolve this situation, try the following:

• View the configuration of the chart and ensure an alertingpolicy is selected. If no alerting policy isdisplayed, then associate the chart with an alerting policy ordelete the chart from the dashboard.

  Charts on dashboards that display data for an alerting policy stopdisplaying data when the referenced alerting policy is deleted.Deleting an alerting policy doesn't delete charts that reference that policy.

• View the details of the alerting policy and verify that itcontains a single condition.

  Charts on dashboards that display data for an alerting policy stop displayingdata if the alerting policy is modified to contain multiple conditions.To resolve this condition,associate the chart with a single-condition alerting policy ordelete the chart from the dashboard.

• Increase the period of time range. The time-range selector must be set to avalue longer than the sampling period of the data. While most metrics aresampled every 60 seconds, some metrics are sampled infrequently.For example, theserviceruntime.googleapis.com/quota/limitmetric is sampled every 86,400 seconds (1 day).

Logs panel reports invalid query

You see an error message "Error: Invalid query", and the logs panel doesn'tshow any log entries.

To resolve this issue, reduce the number of projects whose logs thelogs panel is configured to display.

For more information about this failure condition, seeQuery returns an error.

Logs panel reports no entries found

You see the error message "No entries found matching current filter" ina logs panel. There are multiple reasons why you might see this message.Check the following:

• For the each project whose log entries you want to view, ensure that one ofthe following is true:

  • You have theLogs Viewer(roles/logging.viewer) role.

  • For custom roles, you have the following permissions:

    • logging.logs.list
    • logging.logServiceIndexes.list
    • logging.logServices.list

• Ensure that the clauses in the query editor don't conflict with thesettings shown in the logs panel.

  For example, when theQuery Editor field containsseverity=NOTICE,the logs panel retrieves all logs that have a severity level ofexactlyNOTICE. If the logs panelSeverity menu is set to"Warning", then the logs panel displays"No entries found matching current filter". The message is shown becausenone of the retrieved log entries have a severity level of at least"Warning".

  In contrast, when theQuery Editor field containsseverity=WARNINGand the logs panelSeverity menu is set to "Notice", you mightsee log entries. With this configuration, the logs panel retrieveslogs whose severity level isWARNING. Next, that list of logs is filteredto display only those whose severity is at least "Notice", and all of theretrieved logs have at least that severity level.

Logs panel indicates it displays partial data

You see a warning indicator and the message "Showing partial results" ona logs panel.

When you don't have permission to view logs for a Google Cloud project whose logsyou've selected to display, this indicator is shown. To remove the warningindicator, do one of the following:

• Contact your system administrator and ask them to do one of the followingfor each Google Cloud project whose logs you want to view::

  • Grant you the role ofLogs Viewer(roles/logging.viewer).

  • For custom roles, grant you the following permissions:

    • logging.logs.list
    • logging.logServiceIndexes.list
    • logging.logServices.list

• Change the configuration of the logs panel to show logs only forGoogle Cloud projects where you have permission to view log entries.

Logs panel displays permission denied message

The logs panel widget on a dashboard is displaying the following permissiondenied message:

Error: Permission 'logging.views.access' denied on resource (or it may not exist).

To resolve this problem, do the following:

1. Edit the logs panel widget.

2. Do one of the following:

   • Set theScope by menu toProject and then use theProjectsmenu to selectCurrent project.

   • Remove from theStorage views menu the log views that you don't havepermission to view. For information about IAM rolesrequired to access log views, seeGrant access to a log view.

     When the logs panel widget searches log views, you must havepermission to read log entries in all log views that are listed.Otherwise, an error message is displayed.

3. ClickApply.

Table doesn't display all time series

You configure a table, but it doesn't display all time series.

For performance reasons, tables are configured to display at most300 rows.

API call to create dashboard with a logs panel fails

There are multiple reasons why you might see an error message whenyou create a dashboard with a logs panel:

• When the logs panel doesn't meet the minimum height or width requirements,the following error is shown:

  Unable to save dashboard: Field mosaicLayout.tiles[0] has an invalid value: LogsPanel must have a minimum width of 3 and height of 3.

  To resolve this condition, in the dashboard JSON representation, ensure theheight andwidth fields of the logs panel widget meet the minimumrequirements.

• When thelogsPanel.filter field exceeds2,000 characters, the followingerror is shown:

  Unable to save dashboard: filter cannot be greater than 2000 characters.

  To resolve this condition, shorten the filter.

• When theresourceNames field is missing, you see the following error message:

  Unable to save dashboard: Dashboard is missing required field mosaicLayout.tiles[0].widget.logsPanel.resourceNames.

  To resolve this condition, add aresourceNames field to the dashboardJSON.

• When the syntax for theresourceNames is incorrect, the following error issimilar to the following:

  Unable to save dashboard: Field mosaicLayout.tiles[0].widget.logsPanelWidget.resourceNames has an invalid value of "a-project": resource name must have the format `projects/{project}`."

  To resolve this condition, replacea-project withprojects/a-projectin the dashboard JSON.

• For information about how to resolve syntax errors in thefilter field, seeTroubleshooting: syntax issues.

For an example of a correctly formatted JSON file, seeDashboard with aLogsPanel widget.

Importing Grafana dashboards

For information about using and troubleshooting the dashboard importer, seeImport Grafana dashboards intoCloud Monitoring.

For information about problems with the conversion of thedashboard contents, see the importer'sREADME file.