gcloud dataproc jobs submit

NAME
gcloud dataproc jobs submit - submit Dataproc jobs to execute on a cluster
SYNOPSIS
gcloud dataproc jobs submitCOMMAND[--async][--bucket=BUCKET][--region=REGION][GCLOUD_WIDE_FLAG …]
DESCRIPTION
Submit Dataproc jobs to execute on a cluster.
EXAMPLES
To submit a Hadoop MapReduce job, run:
gclouddataprocjobssubmithadoop--clustermy-cluster--jarmy_jar.jar--arg1arg2

To submit a Spark Scala or Java job, run:

gclouddataprocjobssubmitspark--clustermy-cluster--jarmy_jar.jar--arg1arg2

To submit a PySpark job, run:

gclouddataprocjobssubmitpyspark--clustermy-clustermy_script.py--arg1arg2

To submit a Spark SQL job, run:

gclouddataprocjobssubmitspark-sql--clustermy-cluster--filemy_queries.q

To submit a Pig job, run:

gclouddataprocjobssubmitpig--clustermy-cluster--filemy_script.pig

To submit a Hive job, run:

gclouddataprocjobssubmithive--clustermy-cluster--filemy_queries.q
FLAGS
--async
Return immediately, without waiting for the operation in progress to complete.
--bucket=BUCKET
The Cloud Storage bucket to stage files in. Defaults to the cluster's configuredbucket.
--region=REGION
Dataproc region to use. Each Dataproc region constitutes an independent resourcenamespace constrained to deploying instances into Compute Engine zones insidethe region. Overrides the defaultdataproc/region property valuefor this command invocation.
GCLOUD WIDE FLAGS
These flags are available to all commands:--help.

Run$gcloud help for details.

COMMANDS
COMMAND is one of the following:
flink
Submit a Flink job to a cluster.
hadoop
Submit a Hadoop job to a cluster.
hive
Submit a Hive job to a cluster.
pig
Submit a Pig job to a cluster.
presto
Submit a Presto job to a cluster.
pyspark
Submit a PySpark job to a cluster.
spark
Submit a Spark job to a cluster.
spark-r
Submit a SparkR job to a cluster.
spark-sql
Submit a Spark SQL job to a cluster.
trino
Submit a Trino job to a cluster.
NOTES
These variants are also available:
gcloudalphadataprocjobssubmit
gcloudbetadataprocjobssubmit

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-09-16 UTC.