Route logs to supported destinations

This document explains how to create and manage log sinks, which routelog entries that originate in a Google Cloud project to supported destinations.

Asink performs a write action and therefore it mustbe authorized to write to the destination. When the destination is a log bucketin the same project as the sink, the sink is automaticallyauthorized. For all other destinations, the sink must be attached to aservice account that has been grantedthe permissions required to write data to the destination.

When a service account is required, Cloud Logging automatically creates andmanages it. However, you might need to modify the permissions granted tothe service account. You don't have to use the service account created byLogging. You can create and manage a service accountthat is used by sinks in multiple projects. For more information, seeConfigure log sinks with user-managed service accounts.

Note: If your data is managed through anAssured Workloads environment,then this feature might be impacted or restricted. For information, seeRestrictions and limitations in Assured Workloads.

Overview

This page describes how to create a sink and how to configure the optionsyou might see when using the Google Cloud console or the API.

Sinks belong to a given Google Cloud resource: a Google Cloud project,a billing account, a folder, or an organization. When the resource receives alog entry, every sink in the resource processes the log entry. When alog entry matches the filters of the sink, then the log entry isrouted to the sink's destination.

Typically, sinks only route the log entries that originate in a resource.However, for folders and organizations you can createaggregated sinks,which route log entries from the folder or organization, and theresourcesit contains. This document doesn't discussaggregated sinks. For more information, seeAggregated sinks overview.

To create and manage sinks, you can use the Google Cloud console,the Cloud Logging API, and theGoogle Cloud CLI. We recommend thatyou use the Google Cloud console for the following reasons:

  • You can configure sink destinations as part of the flow to create a sink.
  • You can preview which log entries match the sink's filters.
  • Some authorization steps are simplified.

To learn how to view and manage your log sinks, see theManage sinks section of this document.

Supported destinations

Note: To use the visualization and analysis tools of Cloud Logging or to useError Reporting, you must store your log entries in log buckets.These log buckets don't have to be in the same resource where thelog entries originate. For example, you might configure an aggregated sink toroute log entries to a Google Cloud project, and then configure the sinksin that project to reroute the log entries to local log buckets.

The destination of a sink can be in a different resource than the sink.For example, you can use a log sink to route log entries from one project to alog bucket stored in a different project.

The following destinations are supported:

Google Cloud project

Select this destination when you want the log sinks in thedestination project to reroute your log entries, or when you have createdan intercepting aggregated sink. The log sinks in the project that is thesink destination can reroute the log entries to any supported destinationexcept a project.

Note: This is the only type of destination where log entries are rerouted.For example, if you route log entries from one project to a log bucket inanother project, then those log entries aren't rerouted by the log sinksin the project that stores the log bucket.
Log bucket

Select this destination when you want to store your log data inresources managed by Cloud Logging. Log data stored in log bucketscan be viewed and analyzed using services like the Logs Explorerand Log Analytics.

If you want to join your log data with other business data, then youcan store your log data in a log bucket and create a linkedBigQuery dataset. A linked dataset is a read-only datasetthat can be queried like any other BigQuery dataset.

BigQuery dataset
Select this destination when you want to join your log data withother business data. The dataset you specify must be write-enabled.Don't set the destination of a sink to be a linkedBigQuery dataset. Linked datasets are read-only.
Cloud Storage bucket
Select this destination when you want long-term storage of your log data.The Cloud Storage bucket can be in the same project in which log entriesoriginate, or in a different project. Log entries are stored as JSON files.
Pub/Sub topic
Select this destination when you want to export your log data fromGoogle Cloud and then use third-party integrations like Splunk or Datadog.Log entries are formatted into JSON and then routed toa Pub/Sub topic.

Destination limitations

This section describes destination-specific limitations:

  • If you route log entries to a BigQuery dataset, theBigQuery dataset must be write-enabled. You can't routelog entries to linked datasets, which are read-only.
  • New sinks that route log data to Cloud Storage buckets might takeseveral hours to start routing log entries. These sinks are processed hourly.

  • The following limitations apply when the destination of a log sinkis a Google Cloud project:

    • There is a one-hop limit.
    • Log entries that match the filter of the_Required log sinkare only routed to the_Required log bucket of the destination projectwhen they originate in the destination project.
    • Only aggregated sinks that are in the resource hierarchy of a log entryprocess the log entry.

    For example, assume the destination of a log sink in projectA isprojectB. Then the following are true:

    • Due to the one-hop limit, the log sinks in projectB can't reroutelog entries to a Google Cloud project.
    • The_Required log bucket of projectB only stores log entries thatoriginate in projectB. This log bucket doesn't store any logentries that originate in any other resource, including thosethat originate in projectA.
    • If the resource hierarchy of projectA and projectB differ, thena log entry that a log sink in projectA routes to projectBisn't sent to the aggregated sinks in the resource hierarchy ofprojectB.
    • If projectA and projectB have the same resource hierarchy,then log entries are sent to the aggregated sinks in that hierarchy.If a log entry isn't intercepted by an aggregated sink,then the Log Router sends the log entry to the sinks in projectA.

Before you begin

The instructions in this document describe creating and managing sinks at theGoogle Cloud project level. You can use the same procedure to create a sinkthat routes log entries that originate in an organization, folder, orbilling account.

To get started, do the following:

  1. Enable the Cloud Logging API.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

    Enable the API

  2. Make sure that your Google Cloud project contains log entries that you can see intheLogs Explorer.

  3. To get the permissions that you need to create, modify, or delete a sink, ask your administrator to grant you theLogs Configuration Writer (roles/logging.configWriter) IAM role on your project. For more information about granting roles, seeManage access to projects, folders, and organizations.

    You might also be able to get the required permissions throughcustom roles or otherpredefined roles.

    For information about granting IAM roles, seethe LoggingAccess control guide.

  4. You have a resource in asupported destination orhave the ability to create one.

    To route log entries to a destination, the destination must exist beforeyou create the sink. You can create the destination in anyGoogle Cloud project in any organization.

  5. Before you create a sink, review the limitations that apply for thesink destination. For more information, see theDestination limitations section in this document.

  6. Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, aCloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

      gcloudinit

      If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

    For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Create a sink

This section describes how to create a sink in a Google Cloud project.You can create up to 200 sinks per Google Cloud project.To view the number and volume of log entries that are routed, view thelogging.googleapis.com/exports/ metrics.

You use theLogging query language to create a filterexpression that matches the log entries you want to include. Don't putsensitive information in sink filters. Sink filters are treated as service data.

When a query contains multiple statements,you can either specify how those statements are joined or rely on Cloud Logging implicitlyadding the conjunctive restriction,AND, between the statements. For example,suppose a query or filter dialog contains two statements,resource.type = "gce_instance" andseverity >= "ERROR".The actual query isresource.type = "gce_instance" AND severity >= "ERROR".Cloud Logging supports both disjunctive restrictions,OR, and conjunctiverestrictions,AND. When you useOR statements, we recommend that yougroup the clauses with parentheses.

To create a sink, do the following:

Console

  1. In the Google Cloud console, go to theLog Router page:

    Go toLog Router

    If you use the search bar to find this page, then select the result whose subheading isLogging.

  2. Select the Google Cloud project in which the log entries that youwant to route originate.

    For example, if you want to route your Data Access log entries fromthe project namedProject-A to a log bucket in the project namedProject-B, then selectProject-A.

  3. SelectCreate sink.

  4. In theSink details panel, enter the following details:

    • Sink name: Provide an identifier for the sink; note that after youcreate the sink, you can't rename the sink but you can delete it andcreate a new sink.

    • Sink description (optional): Describe the purpose or use case forthe sink.

  5. In theSink destination panel, select the sink service anddestination by using theSelect sink service menu. Do one of thefollowing:

  6. Specify the log entries to include:

    1. Go to theChoose logs to include in sink panel.

    2. In theBuild inclusion filter field, enter a filter expressionthat matches the log entries you want to include. To learn more aboutthe syntax for writing filters, seeLogging query language.

      If you don't set a filter, all log entries from your selected resourceare routed to the destination.

      For example, to route all Data Access log entries to aLogging bucket, you can use the following filter:

      log_id("cloudaudit.googleapis.com/data_access")ORlog_id("externalaudit.googleapis.com/data_access")

      The length of a filter can't exceed 20,000 characters.

    3. To verify you entered the correct filter, selectPreview logs.The Logs Explorer opens in a new tab with the filterpre-populated.

  7. (Optional) Configure an exclusion filter to eliminate some of theincluded log entries:

    1. Go to theChoose logs to filter out of sink panel.

    2. In theExclusion filter name field, enter a name.

    3. In theBuild an exclusion filter field, enter afilter expression thatmatches the log entries you want to exclude. You can also use thesample functionto select a portion of the log entries to exclude.

      Key Point: If you want your exclusion filter to be disabledwhen the sink is created, then selectDisable after you enteryour filter expression. You can update the sink later to enable theexclusion filter.

    You can create up to 50 exclusion filters persink. Note that the length of a filter can't exceed20,000 characters.

  8. SelectCreate sink.

  9. Grant the service account for the sink the permission to write log entriesto your sink's destination. For more information, seeSet destination permissions.

gcloud

To create a sink, do the following:

  1. Run the followinggcloud logging sinks createcommand:

    gcloud logging sinks createSINK_NAMESINK_DESTINATION

    Before running the command, make the following replacements:

    • SINK_NAME: The name of the log sink. You can't change the name of a sink after you create it.

    • SINK_DESTINATION: The service or project to where you want your log entries routed. SetSINK_DESTINATIONwith the appropriate path, as described inDestination path formats.

      For example, if your sink destination is a Pub/Subtopic, thenSINK_DESTINATION looks like the following:

      pubsub.googleapis.com/projects/PROJECT_ID/topics/TOPIC_ID

    You can also provide the following options:

    • --log-filter : Use this option to set afilter thatmatches the log entries you want to include in your sink. If youdon't provide a value for the inclusion filter, then the this filtermatches all log entries.
    • --exclusion: Use this option to set an exclusion filter forlog entries that you want to exclude your sink from routing.You can also use thesample functionto select a portion of the log entries to exclude. This option canbe repeated; you can create up to 50exclusion filters per sink.
    • --description: Use this option to describe the purpose or use casefor the sink.

    For example, to create a sink to a Logging bucket, yourcommand might look like this:

    gcloud logging sinks create my-sink logging.googleapis.com/projects/myproject123/locations/global/buckets/my-bucket \ --log-filter='logName="projects/myproject123/logs/matched"' --description="My first sink"

    For more information on creating sinks using theGoogle Cloud CLI, see thegcloud logging sinks reference.

  2. If the command response contains a JSON key labeled"writerIdentity",then grant the service account of the sink the permission to write tothe sink destination. For more information,seeSet destination permissions.

    You don't need to set destination permissions when the responsedoesn't contain a JSON key labeled"writerIdentity".

REST

  1. To create a logging sink in your Google Cloud project, useprojects.sinks.create in the Logging API. In theLogSinkobject, provide the appropriate required values in the method requestbody:

    • name: An identifier for the sink. Note that after you create thesink, you can't rename the sink, but you can delete it and create anew sink.

    • destination: The service and destination to where you want yourlog entries routed. To route log entries to a different project,or to a destination thatis in another project, set thedestination field with theappropriate path, as described inDestination path formats.

      For example, if your sink destination is a Pub/Subtopic, then thedestination looks like the following:

      pubsub.googleapis.com/projects/PROJECT_ID/topics/TOPIC_ID

  2. In theLogSink object, provide the appropriate optionalinformation:

    • filter : Set thefilterfield to match the log entries you want to include in your sink. Ifyou don't set a filter, all log entries from yourGoogle Cloud project arerouted to the destination. Note that the length of a filter can'texceed 20,000 characters.
    • exclusions: Set this field to match the log entries that you wantto exclude from your sink. You can also use thesample functionto select a portion of the log entries to exclude. You can create up to50 exclusion filters per sink.
    • description: Set this field to describe the purpose or use case forthe sink.

  3. Callprojects.sinks.create to create the sink.

  4. If the API response contains a JSON key labeled"writerIdentity",then grant the service account of the sink the permission to write tothe sink destination. For more information,seeSet destination permissions.

    You don't need to set destination permissions when the API responsedoesn't contain a JSON key labeled"writerIdentity".

For more information on creating sinks using theLogging API, see theLogSink reference.

If you receive error notifications, then seeTroubleshoot routing and sinks.

Destination path formats

If you route log entries to a service that is in another project, then you mustprovide the sink with the fully-qualified name for the service. Similarly,if you route log entries to a different Google Cloud project, then you mustprovide the sink with the fully-qualified name of the destination project:

  • Cloud Logging log bucket:

    logging.googleapis.com/projects/DESTINATION_PROJECT_ID/locations/LOCATION/buckets/BUCKET_NAME

  • Another Google Cloud project:

    logging.googleapis.com/projects/DESTINATION_PROJECT_ID

Set destination permissions

This section describes how to grant Logging theIdentity and Access Management permissions to write log entries to your sink's destination.For the full list of Logging roles and permissions,seeAccess control.

Note: To secure routed log entries from unauthorized access, you must use theaccess control features of the destination. Sinks can route any log entries,including Access Transparency log entries and Data Access audit log entries.If the routed log entries produce Error Reporting data in thedestination, then you must also configure Error Reportingaccess control features. For more information aboutError Reporting access control, seeControl access with IAM.

Cloud Logging creates a shared service account for a resource when asink is created, unless the required service account already exists.The service account might exist because the same service account is used forall sinks in the underlying resource. Resources can be a Google Cloud project,an organization, a folder, or a billing account.

Thewriter identity of a sink is the identifier of the serviceaccount associated with that sink. All sinks have a writer identity except forsinks that write to a log bucket in the same Google Cloud project in whichthe log entry originates. For the latter configuration, a service accountisn't required and therefore the sink'swriter identity fieldis listed asNone in the console. TheAPI and the Google Cloud CLI commands don't report a writer identity.

The following instructions apply to projects, folders, organizations, andbilling accounts:

Console

Note: If you created your sink in the Google Cloud console and you haveOwner access to the destination, then Cloud Logging should have setup the necessary permissions on your behalf. If not, thencomplete the following steps:

  1. Make sure that you haveOwner access on theGoogle Cloud project that contains the destination.If you don't haveOwner access to the destination of the sink,then ask a project owner to add the writer identity as a principal.

  2. To get the sink's writer identity—an email address—from thenew sink, do the following:

    1. In the Google Cloud console, go to theLog Router page:

      Go toLog Router

      If you use the search bar to find this page, then select the result whose subheading isLogging.

    2. In the toolbar, select the project that contains the sink.
    3. SelectMenu and then selectView sink details. The writer identity appears in theSink details panel.

  3. If the value of thewriterIdentity field contains an email address,then proceed to the next step. When the value isNone,you don't need to configure destination permissions for the sink.

  4. Copy the sink's writer identity into your clipboard.

    The email address identifies the principal. The prefix,serviceAccount:,specifies the account type.

  5. Grant the principal specified in the sink's writer identity the permissionto write log data to the destination:

    1. In the Google Cloud console, go to theIAM page:

      Go toIAM

      If you use the search bar to find this page, then select the result whose subheading isIAM & Admin.

    2. In the toolbar, make sure that the selected project is either theproject that stores the destination or is the sink destination.For example, if the destination is a log bucket, then make sure thatthe toolbar displays the project that stores the log bucket.

    3. ClickGrant access.

    4. Grant the principal specified in the sink's writer identityan IAM role based on the destination of the log sink:

gcloud

  1. Make sure that you haveOwner access on theGoogle Cloud project that contains the destination.If you don't haveOwner access to the destination of the sink,then ask a project owner to add the writer identity as a principal.

  2. Get the service account from thewriterIdentity field in your sink:

    gcloud logging sinks describeSINK_NAME

  3. Locate the sink whose permissions you want to modify, and if the sinkdetails contain a line withwriterIdentity, then proceedto the next step. When the details don't include awriterIdentityfield, you don't need to configure destination permissions forthe sink.

    The writer identity for the service account looks similar to thefollowing:

    serviceAccount:service-123456789012@gcp-sa-logging.iam.gserviceaccount.com

  4. Grant the sink's writer identity the permissionto write log data to the destination by calling thegcloud projects add-iam-policy-binding command.

    Before using the following command, make the following replacements:

    • PROJECT_ID: The identifier of the project. Specify the project which stores the destination ofthe log sink. When the destination is a project, specify that project.
    • PRINCIPAL: An identifier for the principal that you want to grant the role to. Principal identifiers usually have the following form:PRINCIPAL-TYPE:ID. For example,user:my-user@example.com. For a full list of the formats thatPRINCIPAL can have, seePrincipal identifiers.

    • ROLE: An IAM role. Grant the sink's writer identity an IAMrole based on the destination of the log sink:

    Execute thegcloud projects add-iam-policy-bindingcommand:

    gcloudprojectsadd-iam-policy-bindingPROJECT_ID--member=PRINCIPAL--role=ROLE

REST

We recommend that you use the Google Cloud console or the Google Cloud CLIto grant a role to service account.

Manage sinks

After your sinks are created, you can perform the following actions on them.Any changes made to a sink might take a few minutes to apply:

  • View details
  • Update

  • Disable

    • You can't disable the_Required sink.
    • You can disable the_Default sink to stop it from routing log entries tothe_Default Logging bucket.
    • If you want to disable the_Default sink for any newGoogle Cloud projects or folders created in your organization,then consider configuringdefault resource settings.

  • Delete

    • You can't delete the_Default or the_Required sinks.
    • When you delete a sink, it no longer routes log entries.
    • If the sink has a dedicated service account, then deleting that sink alsodeletes the service account. Sinks created beforeMay 22, 2023 have dedicated service accounts. Sinks createdon or after May 22, 2023 have a shared service account.Deleting the sink doesn't delete the shared service account.

  • Troubleshoot failures

  • View log volume and error rates

Following are the instructions for managing a sink in a Google Cloud project.Instead of a Google Cloud project, you can specify a billing account,folder, or organization:

Console

  1. In the Google Cloud console, go to theLog Router page:

    Go toLog Router

    If you use the search bar to find this page, then select the result whose subheading isLogging.

  2. In the toolbar, select the resource that contains your sink. Theresource can be a project, folder, organization, or billing account.

TheLog Router page displays the sinks in the selected resource.Each table row contains information about a sink's properties:

  • Enabled: Indicates if the sink's state is enabled or disabled.
  • Type: The sink's destination service; for example,Cloud Logging bucket.
  • Name: The sink's identifier, as provided when the sink was created;for example_Default.
  • Description: The sink's description, as provided when the sink wascreated.
  • Destination: Full name of the destination to which the routed logentries are sent.
  • Created: The date and time that the sink was created.
  • Last updated: The date and time that the sink was last edited.
  • Volume: Reports the total volume of logs routed tothe log sink. The value includes the volume routed to log buckets,projects, or to other destinations.

For each table row, theMore actions menuprovides the following options:

  • View sink details: Displays the sink's name, description,destination service, destination, and inclusion and exclusion filters.SelectingEdit opens theEdit Sink panel.
  • Edit sink: Opens theEdit Sink panel where you can update thesink's parameters.
  • Disable sink: Lets you disable the sink and stop routing log entriesto thesink's destination. For more information on disabling sinks, seeStop storing logs in log buckets.
  • Enable sink: Lets you enable a disabled sink and restart routinglog entries to the sink's destination.
  • Delete sink: Lets you delete the sink and stop routing log entries tothe sink's destination.
  • Troubleshoot sink: Opens the Logs Explorer where you can troubleshooterrors with the sink.
  • View sink log volume and error rates: Opens the Metrics Explorerwhere you can view and analyze data from the sink.

To sort the table by a column, select the column name.

gcloud

  • To view your list of sinks for your Google Cloud project, use thegcloud logging sinks listcommand, which corresponds to the Logging API methodprojects.sinks.list:

    gcloud logging sinks list

    To view your list ofaggregated sinks, use theappropriate option to specify the resource that contains the sink. Forexample, if you created the sink at the organization level, use the--organization=ORGANIZATION_ID option to list thesinks for the organization.

  • To describe a sink, use thegcloud logging sinks describecommand, which corresponds to the Logging API methodprojects.sinks.get:

    gcloud logging sinks describeSINK_NAME

  • To update a sink, use thegcloud logging sinks updatecommand, which corresponds to the API methodprojects.sink.update.

    You can update a sink to change the destination, filters, anddescription, or to disable or re-enable the sink:

    gcloud logging sinks updateSINK_NAMENEW_DESTINATION --log-filter=NEW_FILTER

    Omit theNEW_DESTINATION or--log-filter if those partsdon't change.

    For example, to update the destination of your sink namedmy-project-sink to a new Cloud Storage bucket destinationnamedmy-second-gcs-bucket, your command looks like this:

    gcloud logging sinks update  my-project-sink storage.googleapis.com/my-second-gcs-bucket

  • To disable a sink, use thegcloud logging sinks updatecommand, which corresponds to the API methodprojects.sink.update,and include the--disabled option:

    gcloud logging sinks updateSINK_NAME --disabled

    To reenable the sink, use thegcloud logging sinks updatecommand, remove the--disabled option, and include the--no-disabledoption:

    gcloud logging sinks updateSINK_NAME --no-disabled

  • To delete a sink, use thegcloud logging sinks deletecommand, which corresponds to the API methodprojects.sinks.delete:

    gcloud logging sinks deleteSINK_NAME

    For more information on managing sinks using theGoogle Cloud CLI, see thegcloud logging sinks reference.

REST

Stop storing log entries in log buckets

Note: If you want to disable the_Default sinks created in yourorganization, then configure thedefault resource settings.

You can disable the_Default sink and any user-defined sinks. When youdisable a sink, the sink stops routing log entries to its destination.For example, if you disable the_Default sink, then no log entries arerouted to the_Default bucket. The_Default bucket becomes empty when all of the previously stored log entrieshave fulfilled the bucket'sretention period.

The following instructions illustrate how todisable your Google Cloud project sinks that route log entries to the_Default log buckets:

Console

  1. In the Google Cloud console, go to theLog Router page:

    Go toLog Router

    If you use the search bar to find this page, then select the result whose subheading isLogging.

  2. To find all the sinks that route log entries to the_Default log bucket,filter the sinks by destination, and then enter_Default.

  3. For each sink, selectMenu andthen selectDisable sink.

    The sinks are now disabled and your Google Cloud project sinks nolonger route log entries to the_Default bucket.

To reenable a disabled sink and restart routing log entries to the sink'sdestination, do the following:

  1. In the Google Cloud console, go to theLog Router page:

    Go toLog Router

    If you use the search bar to find this page, then select the result whose subheading isLogging.

  2. To find all the sinks that route log entries to the_Default log bucket,filter the sinks by destination, and then enter_Default.
  3. For each sink, selectMenu andthen selectEnable sink.

gcloud

  1. To view your list of sinks for your Google Cloud project, use thegcloud logging sinks listcommand, which corresponds to the Logging API methodprojects.sinks.list:

    gcloud logging sinks list

  2. Identify any sinks that are routing to the_Default log bucket.To describe a sink, including seeing the destination name, use thegcloud logging sinks describecommand, which corresponds to the Logging API methodprojects.sinks.get:

    gcloud logging sinks describeSINK_NAME

  3. Run thegcloud logging sinks updatecommand and include the--disabled option. For example, to disable the_Default sink, use the following command:

    gcloud logging sinks update _Default  --disabled

    The_Default sink is now disabled; it no longer routes log entries tothe_Default log bucket.

To disable the other sinks in your Google Cloud project that are routingto the_Default bucket, repeat the previous steps.

To reenable a sink, use thegcloud logging sinks updatecommand, remove the--disabled option, and include the--no-disabledoption:

gcloud logging sinks update _Default  --no-disabled

REST

  1. To view the sinks for your Google Cloud project, call theLogging API methodprojects.sinks.list.

    Identify any sinks that are routing to the_Default bucket.

  2. For example, to disable the_Default sink,set thedisabled field in theLogSink object totrue, and thencallprojects.sink.update.

    The_Default sink is now disabled; it no longer routes log entries tothe_Default bucket.

To disable the other sinks in your Google Cloud project that are routingto the_Default bucket, repeat the previous steps.

To reenable a sink,set thedisabled field in theLogSink object tofalse, and thencallprojects.sink.update.

Code samples

To use client library code to configure sinks in your chosen languages, seeCode samples.

Filter examples

Following are some filter examples that are particularly useful when creatingsinks. For additional examples that might be useful as you build your inclusionfilters and exclusion filters, seeSample queries.

Restore the_Default sink filter

If you edited the filter for the_Default sink, then you might want to restorethis sink to its original configuration. When created, the_Default sink isconfigured with the following inclusion filter and an empty exclusion filter:

  NOT log_id("cloudaudit.googleapis.com/activity") AND NOT \  log_id("externalaudit.googleapis.com/activity") AND NOT \  log_id("cloudaudit.googleapis.com/system_event") AND NOT \  log_id("externalaudit.googleapis.com/system_event") AND NOT \  log_id("cloudaudit.googleapis.com/access_transparency") AND NOT \  log_id("externalaudit.googleapis.com/access_transparency")

Exclude Google Kubernetes Engine container and pod logs

Note: Excluding some Google Kubernetes Engine log entries might prevent you fromidentifying and resolving problems with an application.For more information, seeSupportability.

To exclude Google Kubernetes Engine container and pod log entries forGKE systemnamespaces, use the following filter:

resource.type = ("k8s_container" OR "k8s_pod")resource.labels.namespace_name = ("cnrm-system" OR"config-management-system" OR"gatekeeper-system" OR"gke-connect" OR"gke-system" OR"istio-system" OR"knative-serving" OR"monitoring-system" OR"kube-system")

To exclude Google Kubernetes Engine node log entries for GKEsystemlogNames, use the following filter:

resource.type = "k8s_node"logName:( "logs/container-runtime" OR"logs/docker" OR"logs/kube-container-runtime-monitor" OR"logs/kube-logrotate" OR"logs/kube-node-configuration" OR"logs/kube-node-installation" OR"logs/kubelet" OR"logs/kubelet-monitor" OR"logs/node-journal" OR"logs/node-problem-detector")

To view the volume of Google Kubernetes Engine node, pod, and container log entriesstored in log buckets, use Metrics Explorer:

Exclude Dataflow logs not required for supportability

To exclude Dataflow log entries that aren't required forsupportability, use the following filter:

resource.type="dataflow_step"labels."dataflow.googleapis.com/log_type"!="system" AND labels."dataflow.googleapis.com/log_type"!="supportability"

To view the volume of Dataflow logs stored in log buckets, useMetrics Explorer.

Supportability

Although Cloud Logging lets you exclude log entries and prevent themfrom being stored in a log bucket,you might want to consider keeping log entries that help with supportability.Using these log entries can help you troubleshoot and identify issueswith your applications.

For example, GKE system log entries are useful totroubleshootyour GKE applications and clusters because they aregenerated for events that happen in your cluster. These log entries can help youdetermine if your application code or the underlying GKEcluster is causing your application error. GKE system logsalso include Kubernetes Audit Logging generated by the Kubernetes API Servercomponent, which includes changes made using the kubectl command and Kubernetesevents.

For Dataflow, we recommended that you, at a minimum, write your systemlogs (labels."dataflow.googleapis.com/log_type"="system") and supportabilitylogs (labels."dataflow.googleapis.com/log_type"="supportability") tolog buckets. These logsare essential for developers to observe and troubleshoot their Dataflowpipelines, and users might not be able to use the DataflowJob details page to view job logs.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.