gcloud beta datapipelines pipeline create - creates Data Pipelines Pipeline

gcloud beta datapipelines pipeline create(PIPELINE :--region=REGION)--pipeline-type=PIPELINE_TYPE[--additional-experiments=[ADDITIONAL_EXPERIMENTS,…]][--additional-user-labels=[ADDITIONAL_USER_LABELS,…]][--dataflow-kms-key=DATAFLOW_KMS_KEY][--dataflow-service-account-email=DATAFLOW_SERVICE_ACCOUNT_EMAIL][--disable-public-ips][--display-name=DISPLAY_NAME][--enable-streaming-engine][--flexrs-goal=FLEXRS_GOAL][--max-workers=MAX_WORKERS][--network=NETWORK][--num-workers=NUM_WORKERS][--parameters=[PARAMETERS,…]][--schedule=SCHEDULE][--scheduler-service-account-email=SCHEDULER_SERVICE_ACCOUNT_EMAIL][--subnetwork=SUBNETWORK][--temp-location=TEMP_LOCATION][--template-file-gcs-location=TEMPLATE_FILE_GCS_LOCATION][--template-type=TEMPLATE_TYPE; default="FLEX"][--time-zone=TIME_ZONE][--worker-machine-type=WORKER_MACHINE_TYPE][--transform-name-mappings=[TRANSFORM_NAME_MAPPINGS,…]--[no-]update][--worker-region=WORKER_REGION    |--worker-zone=WORKER_ZONE][GCLOUD_WIDE_FLAG …]

Pipeline resource - Name for the Data Pipelines Pipeline. The arguments in thisgroup can be used to specify the attributes of this resource. (NOTE) Someattributes are not given arguments in this group but can be set in other ways.

To set theproject attribute:

• provide the argumentpipeline on the command line with a fullyspecified name;
• provide the argument--project on the command line;
• set the propertycore/project.

This must be specified.

PIPELINE

ID of the pipeline or fully qualified identifier for the pipeline.

To set thepipeline attribute:

• provide the argumentpipeline on the command line.

This positional argument must be specified if any of the other arguments in thisgroup are specified.

--region=REGION

The Cloud region for the pipeline.

To set theregion attribute:

• provide the argumentpipeline on the command line with a fullyspecified name;
• provide the argument--region on the command line.

--pipeline-type=PIPELINE_TYPE

Type of the pipeline. One of 'BATCH' or 'STREAMING'.PIPELINE_TYPE must be one of:

batch

Specifies a Batch pipeline.

streaming

Specifies a Streaming pipeline.

--additional-experiments=[ADDITIONAL_EXPERIMENTS,…]

Default experiment flags for the job.

--additional-user-labels=[ADDITIONAL_USER_LABELS,…]

Default user labels to be specified for the job. Keys and values must follow therestrictions specified inhttps://cloud.google.com/compute/docs/labeling-resources#restrictions.

--dataflow-kms-key=DATAFLOW_KMS_KEY

Default Cloud KMS key to protect the job resources. The key must be in samelocation as the job.

--dataflow-service-account-email=DATAFLOW_SERVICE_ACCOUNT_EMAIL

Default service account to run the dataflow workers as.

--disable-public-ips

Specifies that Cloud Dataflow workers must not use public IP addresses bydefault. Overrides the defaultdatapipelines/disable_public_ipsproperty value for this command invocation.

--display-name=DISPLAY_NAME

Display name of the Data Pipelines pipeline.

--enable-streaming-engine

Specifies that enabling Streaming Engine for the job by default. Overrides thedefaultdatapipelines/enable_streaming_engine property value forthis command invocation.

--flexrs-goal=FLEXRS_GOAL

FlexRS goal for the flex template job.FLEXRS_GOAL mustbe one of:COST_OPTIMIZED,SPEED_OPTIMIZED.

--max-workers=MAX_WORKERS

Maximum number of workers to run by default. Must be between 1 and 1000.

--network=NETWORK

Default Compute Engine network for launching instances to run your pipeline. Ifnot specified here, defaults to the network 'default'.

--num-workers=NUM_WORKERS

Initial number of workers to run by default. Must be between 1 and 1000. If notspecified here, defaults to server-specified value.

--parameters=[PARAMETERS,…]

User defined parameters for the template.

--schedule=SCHEDULE

Unix-cron format of the schedule for scheduling recurrent jobs.

--scheduler-service-account-email=SCHEDULER_SERVICE_ACCOUNT_EMAIL

Default service account used by the Cloud Scheduler job for launching jobs.

--subnetwork=SUBNETWORK

Default Compute Engine subnetwork for launching instances to run your pipeline.

--temp-location=TEMP_LOCATION

Default Google Cloud Storage location to stage temporary files. If not set,defaults to the value for staging-location (Must be a URL beginning with'gs://'.)

--template-file-gcs-location=TEMPLATE_FILE_GCS_LOCATION

Location of the template file or container spec file in Google Cloud Storage.

--template-type=TEMPLATE_TYPE; default="FLEX"

Type of the template. Defaults to flex template. One of 'FLEX' or 'CLASSIC'.TEMPLATE_TYPE must be one of:

classic

Specifies a Classic template

flex

Specifies a Flex template.

--time-zone=TIME_ZONE

Timezone ID. This matches the timezone IDs used by the Cloud Scheduler API.

--worker-machine-type=WORKER_MACHINE_TYPE

Default type of machine to use for workers. If not specified here, defaults toserver-specified value.

--transform-name-mappings=[TRANSFORM_NAME_MAPPINGS,…]

Transform name mappings for the streaming update job.

--[no-]update

Set this to true for streaming update jobs. Use--update to enableand--no-update to disable.

At most one of these can be specified:

--worker-region=WORKER_REGION

Default Compute Engine region in which worker processing will occur.

--worker-zone=WORKER_ZONE

Default Compute Engine zone in which worker processing will occur.

Run$gcloud help for details.