gcloud alpha dataproc jobs submit - submit Dataproc jobs to execute on a cluster

gcloud alpha dataproc jobs submitCOMMAND[--async][--bucket=BUCKET][--region=REGION][GCLOUD_WIDE_FLAG …]

gcloudalphadataprocjobssubmithadoop--clustermy-cluster--jarmy_jar.jar--arg1arg2

To submit a Spark Scala or Java job, run:

gcloudalphadataprocjobssubmitspark--clustermy-cluster--jarmy_jar.jar--arg1arg2

To submit a PySpark job, run:

gcloudalphadataprocjobssubmitpyspark--clustermy-clustermy_script.py--arg1arg2

To submit a Spark SQL job, run:

gcloudalphadataprocjobssubmitspark-sql--clustermy-cluster--filemy_queries.q

To submit a Pig job, run:

gcloudalphadataprocjobssubmitpig--clustermy-cluster--filemy_script.pig

To submit a Hive job, run:

gcloudalphadataprocjobssubmithive--clustermy-cluster--filemy_queries.q

--async

Return immediately, without waiting for the operation in progress to complete.

--bucket=BUCKET

The Cloud Storage bucket to stage files in. Defaults to the cluster's configuredbucket.

--region=REGION

Dataproc region to use. Each Dataproc region constitutes an independent resourcenamespace constrained to deploying instances into Compute Engine zones insidethe region. Overrides the defaultdataproc/region property valuefor this command invocation.

Run$gcloud help for details.

flink

(ALPHA) Submit a Flink job to a cluster.

hadoop

(ALPHA) Submit a Hadoop job to a cluster.

hive

(ALPHA) Submit a Hive job to a cluster.

pig

(ALPHA) Submit a Pig job to a cluster.

presto

(ALPHA) Submit a Presto job to a cluster.

pyspark

(ALPHA) Submit a PySpark job to a cluster.

spark

(ALPHA) Submit a Spark job to a cluster.

spark-r

(ALPHA) Submit a SparkR job to a cluster.

spark-sql

(ALPHA) Submit a Spark SQL job to a cluster.

trino

(ALPHA) Submit a Trino job to a cluster.