Database observability is a measure of how accurately you can infer the internalstate of a database system based on the data, or telemetry, that it generates inlogs, metrics, and traces.

Diagnosing and troubleshooting issues in an application can be particularlydifficult and time-consuming when a database is involved. Telemetry collectionis crucially important. Telemetry, when enriched with application context, canmake database instances more understandable, observable, and easier to maintain.You can identify issues and problematic trends easily and remedy them early,without having to incur costly downtime. Moreover, by using such data, you canconfigure newer database instances to collect the right kind of data from themoment they start.

You can use data effectively and proactively to prevent issues and focuson strategic innovation. Good telemetry collection is particularly usefulin theDevOps model, where databasegeneralists need to independently analyze telemetry to monitor, evaluate, andoptimize the performance and health of their rapidly evolving applications.

Google Cloud offers several powerful features spanning the four iterativeobservability stages to help you maintain the health of your Cloud SQLdatabase.

Automated telemetry collection

To achieve observability goals, we start by collecting telemetry,preferably through an automated process. When collected over a period,telemetry helps establish a baseline for metrics under different load conditions.

Google Cloud services automatically generate observability data, including metrics,logs, and traces, which can help provide a complete observability overview.

• Cloud Monitoring collects measurementsof your service and of the Google Cloud resources that you use. Cloud SQL usesbuilt-in memory custom agents to collect query telemetry, resulting in a lowerimpact on performance and eliminating the need for agent maintenance or security overhead.

• Cloud Logging collects logging data fromcommon application components. For Cloud SQL, see alsoView instance logs.

• Cloud Trace collects latency data and executedquery plans from applications to help you track how requests propagate throughyour application. You can compare these latency distributions over time oracross versions. Cloud Trace alerts you when it detects asignificant shift in the latency profile of your application when it'sinstrumented to use Cloud Trace.

Sqlcommenter,anOpenTelemetry libraryfor databases helps you monitor your databases through the lens of anapplication. Sqlcommenter automatically instruments ORMs to augment SQLstatements with tags and allows OpenTelemetry trace context information to bepropagated to the database.

With tags and trace application context in databases, it's easy to correlateapplication code with database performance and troubleshoot microservices-basedarchitectures.

Database monitoring

Proper monitoring helps you determine whether your application is working optimally.Implement monitoring early, such as before you initiate a migration ordeploy a new application to a production environment. Disambiguate betweenapplication issues and underlying cloud issues.

TheCloud SQL System Insights dashboardconsiders several important signals of overall database health and performance.

The dashboard shows graphs for several important metrics, whichhelp you gain insights into issues, such as throughput, latency, andcost. These insights help you respond proactively as your application needschange. You can compare current performance against past trends and identifyanomalies that might need investigation.

The Cloud SQLOverview page shows graphs for some of the key metrics.

Cloud SQL also helps youcompare metrics for selected instances.

You can use Cloud Monitoring to createcustom dashboardsthat help you monitor metrics and toset up alert policiesso that you can receive timely notifications.

Database and query analysis

The Cloud SQLQuery Insights toolprovides monitoring and diagnostics that let you detect and fixquery performance problems.

Query Insights dashboards help you identify query performance problems early andlet you move from detection to resolution by using a single interface. Built-in,visual query plans assist you in troubleshooting issues to find the root cause.You can also use in-context, end-to-end application tracing to find the sourceof a problematic query.

Query Insights provides application-centric monitoring that helps youdiagnose performance problems for applications built using object-relationalmappings (ORMs). You can tag queries with business logic that the query isassociated with, such as payment, inventory, business analytics, or shipping.Query Insights can integrate with your existing APM tools, letting you monitorand troubleshoot query problems using your favorite tool.

The Query Insights tool usessqlcommenterto automatically instrument your ORMs. This instrumentation helps you identifythe application code that's causing problems. Query Insights supportsOpenTelemetry standards and makes the query metricsand traces data available for your APM tools through theGoogle Cloud Observability API.

Query Insights integrates withCloud Monitoring,letting you create custom dashboards and alerts on query metrics or tags and receivenotifications using email, SMS, Slack, PagerDuty, and more.

Database tuning

You can iterativelytroubleshoot and tuneyour database.

Cloud SQL recommenders help you analyze the current usage of your databaseand providerecommendationsandinsights based onheuristic methods and machine learning.

Cloud SQL recommenders are briefly described as follows:

What's next

• View the list ofCloud SQL metrics.
• Use Query Insights to improve query performance.
• Use System Insights to improve instance performance.
• View the video:Introducing Cloud SQL insights.
• Read the launch blog:Database observability for developers: introducing Cloud SQL insights.
• Read the blog:Boost your query performance troubleshooting skills with Cloud SQL Insights.
• Read the blog:Enable query tagging with Sqlcommenter.
• Learn more aboutCloud Logging andCloud Monitoring. See alsoView instance logs.
• Troubleshoot and tune your database instance.
• Learn more aboutGoogle Cloud recommenders.