gcloud storage batch-operations jobs create - create a new batch operation job

gcloud storage batch-operations jobs createBATCH_JOB--bucket=BUCKET(--included-object-prefixes=[PREFIXES,…]    |--manifest-location=MANIFEST_LOCATION)(--put-metadata=KEY=VALUE,[KEY=VALUE,…]    |--rewrite-object=KEY=VALUE,[KEY=VALUE,…]    | [--delete-object :--enable-permanent-object-deletion]    |--[no-]put-object-event-based-hold--[no-]put-object-temporary-hold)[--description=DESCRIPTION][--dry-run][--log-actions=[LOG_ACTIONS,…]--log-action-states=[LOG_ACTION_STATES,…]][GCLOUD_WIDE_FLAG …]

gcloudstoragebatch-operationsjobscreatemy-dry-run-job--bucket=my-bucket--manifest-location=gs://my-bucket/manifest.csv--delete-object--dry-run

The following example command creates a batch job, namedmy-job,that performs object deletion on bucketmy-bucket for objectsspecified in the manifest filegs://my-bucket/manifest.csv:

gcloudstoragebatch-operationsjobscreatemy-job--bucket=my-bucket--manifest-location=gs://my-bucket/manifest.csv--delete-object

The following example command creates a batch job, namedmy-job,that updates object metadataContent-Disposition toinline,Content-Language toen, and setsobject retention mode tolocked on bucketmy-bucketfor objects with prefixesprefix1 orprefix2:

gcloudstoragebatch-operationsjobscreatemy-job--bucket=my-bucket--included-object-prefixes=prefix1,prefix2--put-metadata=Content-Disposition=inline,Content-Language=en,Retain-Until=2025-01-01T00:00:00Z,Retention-Mode=locked

The following example command creates a batch job, namedmy-job,that puts object event based hold on objects in bucketmy-bucketwith logging config enabled for corresponding transform action and succeeded andfailed action states:

gcloudstoragebatch-operationsjobscreatemy-job--bucket=my-bucket--put-object-event-based-hold--put-metadata=Content-Disposition=inline,Content-Language=en--log-actions=transform--log-action-states=succeeded,failed

Batch job resource - The batch job to create. This represents a Cloud resource.(NOTE) Some attributes are not given arguments in this group but can be set inother ways.

To set theproject attribute:

• provide the argumentbatch_job on the command line with a fullyspecified name;
• provide the argument--project on the command line;
• set the propertycore/project.

To set thelocation attribute:

• provide the argumentbatch_job on the command line with a fullyspecified name;
• The default isglobal.

This must be specified.

BATCH_JOB

ID of the batch-job or fully qualified identifier for the batch-job.

To set thebatch-job attribute:

• provide the argumentbatch_job on the command line.

--bucket=BUCKET

Bucket containing the objects that the batch job will operate on.

Source specifying objects to perform batch operations on. Must be one of--manifest-location=``MANIFEST_LOCATION or--included-object-prefixes=``COMMA_SEPARATED_PREFIXES

Exactly one of these must be specified:

--included-object-prefixes=[PREFIXES,…]

A comma-separated list of object prefixes to describe the objects beingtransformed. An empty string means all objects in the bucket.

--manifest-location=MANIFEST_LOCATION

An absolute path to the manifest source file in a Google Cloud Storage bucket.The file must be a CSV file where each row specifies the object details i.e.ProjectId, BucketId, and Name. Generation may optionally be specified. Whengeneration is not specified, the live object is acted upon. Format:--manifest-location=gs://bucket_name/path/manifest_name.csv

Transformation to be performed on the objects.

Exactly one of these must be specified:

--put-metadata=KEY=VALUE,[KEY=VALUE,…]

Sets object metadata. To set how content should be displayed, specify the thekey-value pairContent-Disposition={VALUE}. To set how content isencoded (e.g. "gzip"), specify the key-value pairContent-Encoding={VALUE}. To set content's language (e.g. "en"signifies "English"), specify the key-value pairContent-Language={VALUE}. To set the type of data contained in theobject (e.g. "text/html"), specify the key-value pairContent-Type={VALUE}. To set how caches should handle requests andresponses, specify the key-value pairCache-Control={VALUE}. To setcustom time for Cloud Storage objects in RFC 3339 format, specify the key-valuepairCustom-Time={VALUE}. To set object retention, specifyRetain-Until={TIMESTAMP} in RFC 3339 format andRetention-Mode={MODE} where mode can beLocked orUnlocked. To set custom metadata on objects, specify key-valuepairs{CUSTOM-KEY}:{VALUE}. Note that all predefined keys (e.g.Content-Disposition) are case-insensitive. Any other key that is not specifiedabove will be treated as a custom key. To clear a field, provide the key with anempty value (e.g.Content-Disposition=). Multiple key-value pairscan be specified by separating them with commas. For example,--put-metadata=Content-Disposition=inline,Content-Encoding=gzip

--rewrite-object=KEY=VALUE,[KEY=VALUE,…]

Rewrites object and the specified metadata. Currently only supports rewritingkms-key. A metadata field MUST be specified. For example,--rewrite-object=kms-key=projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/CRYPTO_KEYwill rewrite the Cloud KMS key that will be used to encrypt the object.

Or at least one of these can be specified:

Describes options to delete objects.

--delete-object

If this flag is set, objects specified in source will be deleted. Whenversioning is enabled on the buckets, live objects in versioned buckets willbecome noncurrent and objects that were already noncurrent will be skipped.

This flag argument must be specified if any of the other arguments in this groupare specified.

--enable-permanent-object-deletion

If this flag is set and versioning is enabled on the buckets, both live andnoncurrent objects will be permanently deleted.

Or at least one of these can be specified:

Describes options to update object hold.

--[no-]put-object-event-based-hold

Sets or unsets object event based holds state. When object event based hold isset, object cannot be deleted or replaced. Use--put-object-event-based-hold to enable and--no-put-object-event-based-hold to disable.

--[no-]put-object-temporary-hold

Sets or unsets object temporary holds state. When object temporary hold is set,object cannot be deleted or replaced. Use--put-object-temporary-hold to enable and--no-put-object-temporary-hold to disable.

--description=DESCRIPTION

Description for the batch job.

--dry-run

If true, the job will run in dry run mode, returning the total object count and,if the object configuration is a prefix list, the bytes found from source. Notransformations will be performed.

LOGGING CONFIG

Configure which transfer actions and action states are reported when logs aregenerated for this job. Logs can be viewed by running the following command:gcloud logging read "resource.type=storagebatchoperations.googleapis.com/Job"

--log-actions=[LOG_ACTIONS,…]

Define the batch job actions to report in logs. (e.g., --log-actions=transform).LOG_ACTIONS must be (only one value is supported):transform.

--log-action-states=[LOG_ACTION_STATES,…]

The states in which the actions specified in --log-actions are logged. Separatemultiple states with a comma, omitting the space after the comma (e.g.,--log-action-states=succeeded,failed).LOG_ACTION_STATESmust be one of:failed,succeeded.

Run$gcloud help for details.