Compute Engine lets you export detailed reports of yourCompute Engine usage to aCloud Storage bucketusing the usage export feature. Usage reports provide information about thelifetime of your resources. For example, you can see how many VM instances inyour project are running ann2-standard-4 machine type and how long eachinstance has been running. You can also review the storage space of a persistentdisk, and information about other Compute Engine features.

Usage reports don't provide billing or activity information, such asinformation about API requests. For billing information, see theExport Cloud Billing data to BigQueryfeature. For a record of administrative activities and accesses within yourGoogle Cloud resources, seeAudit logs.

Before you begin

• If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

     gcloudinit

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.
  2. Set a default region and zone.

  Python

  To use the Python samples on this page in a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.

  Install the Google Cloud CLI.

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.

  If you're using a local shell, then create local authentication credentials for your user account:

  gcloudauthapplication-defaultlogin

  You don't need to do this if you're using Cloud Shell.

  If an authentication error is returned, and you are using an external identity provider (IdP), confirm that you have signed in to the gcloud CLI with your federated identity.

  For more information, see Set up authentication for a local development environment.

Overview

When you enable usage reports, Compute Engine delivers two types ofreports to the Cloud Storage bucket you specify:

1. Daily usage reports

   These reports are delivered daily and include usage data from the preceding day. Each report is a separate file that contains data from the last period. Data in these reports are immutable, meaning that Compute Engine does not update or rewrite the log file if there are inaccuracies. Instead, the data is corrected in the next new report that is delivered to the bucket.

   Daily usage reports have the following name format:

    <bucket>/<reportprefix><numeric_projectid><YYYYMMDD>.csv

2. Monthly rollup report

   A single monthly rollup report is delivered daily, which contains monthly usage data for that project up to, but not including, that day. The monthly usage report is overwritten each day with new data that reflects the monthly usage of resources up to that date. There is only one monthly usage data file per project, per month.

   Monthly rollup reports have following name format:

    <bucket>/<reportprefix><numeric_projectid><YYYYMM>.csv

   The daily and monthly report files look very similar, except for the difference in date format, where the monthly rollup reports are dated using the year and month (YYYYMM), and the daily usage reports are dated using the year, month, and date (YYYYMMDD).

All usage reports are delivered incomma-separated values (CSV) formatand usage report files are prefixed using<report_prefix>. The<report_prefix> is a customizable value chosen by the user. If you don'tspecify a report prefix, the prefixusage_gce is used by default. All timesare given in Pacific time (PST).

Prerequisites

Before you can start using Compute Engine usage export:

• You must have alreadycreated a bucket tostore usage logs.

Setting up usage export

When you first enable the usage export feature, the first report is sentthe following day, detailing the previous day's usage. Afterwards, youreceive reports in 24 hour intervals.

When you enable this feature, you must define two properties:

1. The Cloud Storagebucket where you would like your reports to be delivered.

   You can select any Cloud Storage bucket for which you are an owner, including buckets that are from different projects. This bucket must exist before you can start exporting reports and you must have owner access to the bucket. Cloud Storage charges for usage, so you should reviewCloud Storage pricing for information on how you might incur charges for the service.

   Any user who has read access to the Cloud Storage bucket can view the usage reports in the bucket. Any user who has write access to the bucket can create, view, and modify existing files in the bucket. For more information, see theAccess control section.

2. The report prefix for your files.

   You can specify the report prefix to use for your usage reports. Your usage reports then have filenames that contain this prefix. For example, specifying "my-cool-project-report" as your report prefix results in a filename similar to the formatmy-cool-project-report_1234567890_20131230.csv. If you don't specify a report prefix, the default prefixusage_gce is used.

After you decide on these two properties, you can enable the usage exportfeature in the following ways:

gcloud compute project-info set-usage-bucket --bucketBUCKET_NAME --prefixPREFIX

from__future__importannotationsimportsysfromtypingimportAnyfromgoogle.api_core.extended_operationimportExtendedOperationfromgoogle.cloudimportcompute_v1defwait_for_extended_operation(operation:ExtendedOperation,verbose_name:str="operation",timeout:int=300)->Any:"""    Waits for the extended (long-running) operation to complete.    If the operation is successful, it will return its result.    If the operation ends with an error, an exception will be raised.    If there were any warnings during the execution of the operation    they will be printed to sys.stderr.    Args:        operation: a long-running operation you want to wait on.        verbose_name: (optional) a more verbose name of the operation,            used only during error and warning reporting.        timeout: how long (in seconds) to wait for operation to finish.            If None, wait indefinitely.    Returns:        Whatever the operation.result() returns.    Raises:        This method will raise the exception received from `operation.exception()`        or RuntimeError if there is no exception set, but there is an `error_code`        set for the `operation`.        In case of an operation taking longer than `timeout` seconds to complete,        a `concurrent.futures.TimeoutError` will be raised.    """result=operation.result(timeout=timeout)ifoperation.error_code:print(f"Error during{verbose_name}: [Code:{operation.error_code}]:{operation.error_message}",file=sys.stderr,flush=True,)print(f"Operation ID:{operation.name}",file=sys.stderr,flush=True)raiseoperation.exception()orRuntimeError(operation.error_message)ifoperation.warnings:print(f"Warnings during{verbose_name}:\n",file=sys.stderr,flush=True)forwarninginoperation.warnings:print(f" -{warning.code}:{warning.message}",file=sys.stderr,flush=True)returnresultdefset_usage_export_bucket(project_id:str,bucket_name:str,report_name_prefix:str="")->None:"""    Set Compute Engine usage export bucket for the Cloud project.    This sample presents how to interpret the default value for the    report name prefix parameter.    Args:        project_id: project ID or project number of the project to update.        bucket_name: Google Cloud Storage bucket used to store Compute Engine            usage reports. An existing Google Cloud Storage bucket is required.        report_name_prefix: Prefix of the usage report name which defaults to an empty string            to showcase default values behaviour.    """usage_export_location=compute_v1.UsageExportLocation()usage_export_location.bucket_name=bucket_nameusage_export_location.report_name_prefix=report_name_prefixifnotreport_name_prefix:# Sending an empty value for report_name_prefix results in the# next usage report being generated with the default prefix value# "usage_gce". (ref: https://cloud.google.com/compute/docs/reference/rest/v1/projects/setUsageExportBucket)print("Setting report_name_prefix to empty value causes the report ""to have the default prefix of `usage_gce`.")projects_client=compute_v1.ProjectsClient()operation=projects_client.set_usage_export_bucket(project=project_id,usage_export_location_resource=usage_export_location)wait_for_extended_operation(operation,"setting GCE usage bucket")

Downloading usage export reports

After you start receiving usage reports in your bucket, download your reportslike you would download other objects from Cloud Storage. For moreinformation, seeDownload objects.

Supported metrics

Daily usage reports provide usage information about the following resources:

• Virtual machines
• Persistent disks
• Images
• Snapshots
• Static IP addresses
• Load balancers
• Reservations

Each resource is described using the following metrics:

cloud-cluster-analytics-export@google.com

Any user who is an owner of the project has full access to theCloud Storage bucket. Other users, such as writers and readers, havedifferent degrees of access to the bucket. To learn about IAMfor a bucket, readIAM for Cloud Storage.

If you disable the usage export feature, Compute Engine automaticallyremoves write access from Compute Engine to the bucket.If you modify the permissions on thecloud-cluster-analytics-export@google.comaccount and then disable the usage export feature, Compute Enginedisables the usage export feature but doesn't remove the account from theproject access list. You can remove the account manually.

Checking if usage reports are enabled

You can check on a project's usage export settings by gettinginformation about the project:

gcloud compute project-info describe

Look for theusageExportLocation field:

+-------------------------+----------------------------------------------------+| name                    | myproject                                          || description             |                                                    || creation-time           | 2019-10-18T16:31:52.308-07:00                      || usage                   |                                                    ||   snapshots             | 1.0/1000.0                                         ||   networks              | 2.0/2.0                                            ||   firewalls             | 3.0/10.0                                           ||...                      |                                                    || usageExportLocation     |                                                    ||   bucketName            | https://storage.googleapis.com/usage-export-sample ||   reportNamePrefix      |                                                    |+-------------------------+----------------------------------------------------+

Disabling usage reports

When you disable usage reports, Compute Engine automatically removeswrite access for Compute Engine to your Cloud Storage bucketand stops sending new reports.

gcloud compute project-info set-usage-bucket --no-bucket

from__future__importannotationsimportsysfromtypingimportAnyfromgoogle.api_core.extended_operationimportExtendedOperationfromgoogle.cloudimportcompute_v1defwait_for_extended_operation(operation:ExtendedOperation,verbose_name:str="operation",timeout:int=300)->Any:"""    Waits for the extended (long-running) operation to complete.    If the operation is successful, it will return its result.    If the operation ends with an error, an exception will be raised.    If there were any warnings during the execution of the operation    they will be printed to sys.stderr.    Args:        operation: a long-running operation you want to wait on.        verbose_name: (optional) a more verbose name of the operation,            used only during error and warning reporting.        timeout: how long (in seconds) to wait for operation to finish.            If None, wait indefinitely.    Returns:        Whatever the operation.result() returns.    Raises:        This method will raise the exception received from `operation.exception()`        or RuntimeError if there is no exception set, but there is an `error_code`        set for the `operation`.        In case of an operation taking longer than `timeout` seconds to complete,        a `concurrent.futures.TimeoutError` will be raised.    """result=operation.result(timeout=timeout)ifoperation.error_code:print(f"Error during{verbose_name}: [Code:{operation.error_code}]:{operation.error_message}",file=sys.stderr,flush=True,)print(f"Operation ID:{operation.name}",file=sys.stderr,flush=True)raiseoperation.exception()orRuntimeError(operation.error_message)ifoperation.warnings:print(f"Warnings during{verbose_name}:\n",file=sys.stderr,flush=True)forwarninginoperation.warnings:print(f" -{warning.code}:{warning.message}",file=sys.stderr,flush=True)returnresultdefdisable_usage_export(project_id:str)->None:"""    Disable Compute Engine usage export bucket for the Cloud Project.    Args:        project_id: project ID or project number of the project to update.    """projects_client=compute_v1.ProjectsClient()# Setting `usage_export_location_resource` to an# empty object will disable the usage report generation.operation=projects_client.set_usage_export_bucket(project=project_id,usage_export_location_resource={})wait_for_extended_operation(operation,"disabling GCE usage bucket")

What's next

• Turn on theExport Cloud Billing data to BigQuery featureto view your billing logs.
• Track the activity in your project usingCloud Audit Logs.
• Learn more aboutCloud Storage buckets.
• Use thepricing calculator to get an estimated price.