As an alternative to running a workflow on amanaged cluster, you can use acluster selector to choose an existing cluster for your workflow. At theconclusion of the workflow, the selected cluster is not deleted.

Selectors specify one or moreDataproc user labels.Clusters in sameregion as the workflow whose labelsmatch all of the selector labels are eligible to run workflow jobs. If multipleclusters match the selector, Dataproc willchoose the cluster with the most free YARN memory.

Add a cluster selector to a template

You can add a cluster selector to a workflow template using theGoogle Cloud CLI or the Dataproc API.

gcloud dataproc workflow-templates set-cluster-selectortemplate-id \    --region=region \    --cluster-labels=name=value[[,name=value]...]

Use automatically applied labels

You can point a cluster selector to an existing cluster by usingone of the followingautomatically-applied cluster labels:

• goog-dataproc-cluster-name
• goog-dataproc-cluster-uuid

Example:

gcloud dataproc workflow-templates set-cluster-selectortemplate-id \    --region=region \    --cluster-labels=goog-dataproc-cluster-name=my-cluster

Select from a cluster pool

You can let Dataproc choose a cluster from a pool of clusters.The cluster pools can be defined with labels.

Example:

gcloud dataproc clusters create cluster-1 --labels cluster-pool=pool-1 \    --region=regiongcloud dataproc clusters create cluster-2 --labels cluster-pool=pool-1 \    --region=regiongcloud dataproc clusters create cluster-3 --labels cluster-pool=pool-2 \    --region=region

After cluster creation ...

gcloud dataproc workflow-templates create my-template \    --region=regiongcloud dataproc workflow-templates set-cluster-selector my-template \    --region=region \    --cluster-labels=cluster-pool=pool-1

The workflow will be run on either cluster-1 or cluster-2, but not on cluster-3.