This document describes how to set up a Google Cloud project and aCompute Engine virtual machine (VM).

Set up a Google Cloud project

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Google Cloud Dataproc API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the API

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

Toinitialize the gcloud CLI, run the following command:

gcloudinit

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Google Cloud Dataproc API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the API

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

Toinitialize the gcloud CLI, run the following command:

gcloudinit

Note: When you enable the Dataproc API, the DataprocControl API is also enabled. Disabling the Dataproc API doesn'tdisable the Dataproc Control API. You must separately disablethe Dataproc Control API.

Do you Need Credentials? You need credentials only ifyou plan to directly call Dataproc APIs from your application or froma browser. If you plan on using the Google Cloud console or theGoogle Cloud CLI tocreate and manage clusters and to submit jobs, you only need to be logged into the SDK with thegcloud init command.

Optional: Set up a VM

If you plan to rungcloud CLI from aCompute Engine virtual machine (VM) instance, the VM must be set upto access Google Cloud services.

To test whether your VM is set up correctly, rungcloud compute instances describeYOUR_VM_INSTANCE_NAMEfrom the command line. A service account must be listed that includes thehttps://www.googleapis.com/auth/cloud-platform scope. For more informationon setting up a VM, seeCreate a VM that uses a user-managed service account.

Use Cloud Shell to rungcloud CLI commands.As an alternative to setting up your own VM to run SDK commands,open aCloud Shell session toautomatically provision a VM with the necessary scopes, and then rungcloud CLI commands from Cloud Shell.

---