Inventory reports help you manage your objectstorage at scale. It is a faster and scheduled alternative to theObjects: list API operation. Use inventory reports ifyou want to validate the migration of large buckets without any performanceimpact or check the integrity of objects in a single bucket.

Inventory reports contain metadata information about your objects, such as theobject's storage class, ETag, and content type. This information helps you analyzeyour storage costs, audit and validate your objects, and ensure data securityand compliance. You can export inventory reports as comma-separated value (CSV)or Apache Parquet files so you can further analyze it using tools such asBigQuery.

This page provides an overview of inventory reports. For instructions on how touse inventory reports, seeCreate and manage inventory reports.

Overview of inventory reports

Inventory reports contain a list of objects and their associated metadata fora given bucket, also known as thesource bucket. To generate inventoryreports, you must first create aninventory report configuration that defineshow often the reports are generated, the metadata fields you want the reports toinclude, and a bucket to generate and store the reports, also known as thedestination bucket.

When you create an inventory report configuration, it automatically getsassigned a universally unique identifier (UUID). This field is non-editable.However, you can edit the following fields of an inventory reportconfiguration:

• The display name of the inventory report configuration
• The object metadata fields that are included in the inventory reports
• The destination bucket that stores the inventory reports
• The schedule that determines how often inventory reports are generated
• The file format in which inventory reports are generated (CSV or ApacheParquet)

When you delete an inventory report configuration, new inventory reports are nolonger generated for the configuration, but existing inventory reportsremain.

When to use inventory reports

Inventory reports are designed for quick analysis of individual buckets. You can usethem to do the following:

• List all objects within a bucket

• Validate the success of data transfers

• Generate audit reports for a particular bucket

When not to use inventory reports

Manually collating and analyzing inventory reports from many buckets andprojects can be challenging, especially for large-scale analysis. For use casessuch as organization-wide visibility, security analysis, or cost management, youcan useStorage Insights datasets. Storage Insights datasets provideflexibility to configure a custom scope either at the bucket, folder, project,or organization level. Storage Insights datasets also provide additional objectinsights like operational data, error information, aggregated bucket and projectactivity data, and custom metadata.

For Storage Insights datasets, data is refreshed daily for metadata views andevery 4-6 hours for activity views. You can analyze the data using SQL inBigQuery or natural language questions with Gemini.

Use Storage Insights datasets if your goals are as follows:

• Cross-organizational data discovery
• Analysis for cost optimization and lifecycle management
• Time-series analysis to identify trends
• Troubleshoot operations
• Monitor activity occurring on buckets and projects

Storage Insights datasets are an exclusive feature only available through theStorage Intelligence subscription.

Source and destination buckets

Thesource bucket contains the objects for which you want to generateinventory reports. It also contains the inventory report configuration. You canhave up to 100 inventory report configurations in a source bucket.

Thedestination bucket stores the generated inventory reports. The destinationbucket must meet the following requirements:

• It must be in the samelocation as the source bucket.
• It must be in the same project as the source bucket.
• It can be the same as the source bucket.

When you first create an inventory report configuration, a service agent isautomatically created on your behalf. In order to create inventory reportconfigurations and write inventory reports to the destination bucket, both youand your service agent must have required IAM permissions.See required permissionsfor yourself and for yourservice agent.

Inventory reports use the source and destination buckets' names to determinewhich buckets to use when running jobs. If you delete a source or destinationbucket and later create a new bucket with the same name, inventory reportswill run jobs using the new bucket.

Object metadata fields

The following metadata fields can be included in an inventory report.Metadata fields marked as "Required" must be included in the inventory report.

For more information about object metadata fields, seeObject metadata.

Inventory report shards

When an inventory report contains more than 1,000,000 objects,one or more shard objects are generated to compose the inventory report. When allthe shards of an inventory report have been successfully generated, a manifestfile is generated in the same destination bucket as the shards.

Inventory report manifest file

The presence of a manifest file indicates that all the shards composing aninventory report have been generated. The manifest file also provides the namesof the inventory report shard objects.

The manifest file follows the naming conventionREPORT_CONFIG_UUID_TARGET_DATETIME_manifest.json, where:

• REPORT_CONFIG_UUID is the auto-generated UUID of theinventory report configuration.

• TARGET_DATETIME is the auto-generated UTC date andtime at which an inventory report is generated.

An example manifest filename isfc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json.

Themanifest.json file contains the following auto-populated fields:

{"report_config":REPORT_CONFIG_FIELDS,"records_processed":NUMBER_OF_INCLUDED_OBJECTS,"snapshot_time":"SNAPSHOT_TIME,  "target_datetime": "TARGET_DATETIME,"shard_count":SHARD_COUNT,"report_shards_file_names":[SHARD_FILE_NAME...]}

Where:

• REPORT_CONFIG_FIELDS includes fields contained withintheReportConfig resource.

• NUMBER_OF_INCLUDED_OBJECTS is the number of objectsincluded in the inventory report.

• SNAPSHOT_TIME is the auto-generated UTC datetime atwhich the data snapshot occurs. All the data in an inventory report iscaptured at the snapshot.

• TARGET_DATETIME is the auto-generated UTC datetime atwhich an inventory report is generated.

• SHARD_COUNT is the total number of generated shardsthat compose the inventory report.

• SHARD_FILE_NAME is the name of a shard thatcomposes an inventory report.

An examplemanifest.json file looks like the following:

{"report_config":{"name":"projects/123456789098/locations/us/reportConfigs/fcec5187-afa6-48b0-938a-543d16493dc0","createTime":"2023-06-08T08:07:53.397366139Z","updateTime":"2023-06-08T08:07:53.552347723Z","frequencyOptions":{"frequency":"DAILY","startDate":{"year":2023,"month":6,"day":9}"endDate":{"year":2023,"month":6,"day":23}},"csvOptions":{"recordSeparator":"\n","delimiter":","},"objectMetadataReportOptions":{"metadataFields":["project","bucket","name","location","updated","storageClass",],"storageFilters":{"bucket":"my-test-bucket"},"storageDestinationOptions":{"bucket":"example-bucket","destinationPath":"folder/subfolder"}}},"records_processed":3993900,"snapshot_time":"2023-06-06T00:07:27Z","target_datetime":{"year":2023,"month":6,"day":6},"shard_count":4,"report_shards_file_names":["fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_0.csv","fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_1.csv",...],}

Pricing

Using inventory reports is subject to pricing, which is dependent on thebucket's location. For more information about inventory reports pricing, seePricing.

Audit logging

Cloud Storage createsaudit logs when inventory reports aregenerated in the destination bucket. Cloud Storage generates audit logswhenever inventory report configurations are created, updated, or deleted.

Cloud Storagedoes not create audit logs when an inventory reportconfiguration reads object metadata from a source bucket.

Integration with VPC Service Controls

You can provide an additional layer of security for inventory reportsresources by using VPC Service Controls. When you use VPC Service Controls, youadd projects to service perimeters that protect resources and services fromrequests that originate from outside of the perimeter. To learn more aboutVPC Service Controls and service perimeters, seeService perimeter details and configuration.

Limitations

Inventory reports has the following limitations.

• EnablingIP filtering on Cloud Storage buckets restrictsinventory reports from being able to access the bucket, regardless ofwhether they use aservice agent to interact with Cloud Storage.To prevent service disruptions, we recommend not using IP filtering onCloud Storage buckets if you are creating inventory reports for thatbucket.

• Inventory reports aren't supported in the following locations:

  • Regions:

    • africa-south1
    • asia-southeast3
    • europe-north2
    • europe-west10
    • europe-west12
    • me-central1
    • me-central2
    • northamerica-south1

  • Dual-regions:

    • eur5
    • eur7
    • eur8

What's next

Learn how tocreate an inventory report configuration and start generatinginventory reports.