Access control with IAM
This page describes access control options in Cloud Data Fusion.
You can control access to resources in Cloud Data Fusion in the followingways:
To control access forcontrol plane operations, such as creating and updatinginstances through the Google Cloud console,Google Cloud CLI, andREST API, useIdentity and Access Management (IAM), as described on this page.
To grant access to Google Cloud data services, such as BigQuery orCloud Storage to a service account where pipelines are running, useIAM.
To control granular permissions for actions performed in theinstance, known asdata plane operations, such as starting pipelines, use role-based access control(RBAC).
Note: RBAC roles aren't typically assigned to pipeline service accounts.
For information about the architecture and resources involved inCloud Data Fusion access control, seeNetworking.For information about granting roles and permissions, seeManage access to projects, folders, and organizations.
About IAM in Cloud Data Fusion
You control access to Cloud Data Fusion features by grantingIAM roles and permissions toservice accounts and other principals in your Google Cloud project.
To grant fine-grained access to user accounts so that they can use theCloud Data Fusion web interface, useRBAC.
Key Point: You control access for Cloud Data Fusion at theproject level.For example, you can grant access to all Cloud Data Fusion resources withina project to a group of developers.By default, Cloud Data Fusion uses the following service accounts:
Cloud Data Fusion Service Account
The Cloud Data Fusion Service Account is a Google-managed service agentthat can access customer resources at pipeline design time. This service agentis automatically added to a project when you enable theCloud Data Fusion API. It's used for all instances in your project.
The service agent has the following responsibilities:
Communicating with other services, such as Cloud Storage,BigQuery, or Datastream during pipeline design.
Enabling execution by provisioning Dataproc clusters andsubmitting pipeline jobs.
Roles for the Cloud Data Fusion Service Account
By default, the Cloud Data Fusion service account has only theCloud Data Fusion API Service Agent role(roles/datafusion.serviceAgent).
The principal name for this service agent isservice-CUSTOMER_PROJECT_NUMBER@gcp-sa-datafusion.iam.gserviceaccount.com.
The following default resources are associated with the Cloud Data Fusion APIService Agent role.
| Role | Resource | Permissions |
|---|---|---|
| Cloud Data Fusion API Service Agent | Associated services:
| See theCloud Data Fusion API Service Agent permissions. |
Compute Engine default service account or custom service account
The Compute Engine service account is the default account thatCloud Data Fusion uses to deploy and run jobs that access otherGoogle Cloud resources. By default, it attaches to aDataproc cluster VM to let Cloud Data Fusion accessDataproc resources during a pipeline run.
You can choose a custom service account to attach to the Dataproccluster when creating a Cloud Data Fusion instance or by creating newCompute Profiles in the Cloud Data Fusion web interface.
For more information, seeService accounts in Cloud Data Fusion.
Roles for the Compute Engine service account
By default, to access resources (such as sources and sinks) when you run apipeline, Cloud Data Fusion uses theCompute Engine default service account.
Caution: If your instance uses theCompute Engine default service account, don't remove roles. Removing them might cause problems with other Google Cloud services.You can set up auser-managed custom service account forCloud Data Fusion instances and grant a role to this account. Afterwards,you can choose this service account when creating new instances.
Note: If you launch Dataproc clusters in a different Google Cloud project,grant the roles in the project where Dataproc is running. By default, you grant them in the project containing the Cloud Data Fusion instance.Cloud Data Fusion Runner role
In the project containing the Cloud Data Fusion instance, for both defaultand user-managed custom service accounts, grant the Cloud Data Fusion Runnerrole (datafusion.runner).
| Role | Description | Permission |
|---|---|---|
Data Fusion Runner(datafusion.runner) | Lets the Compute Engine service account communicate with Cloud Data Fusion services in thetenant project | datafusion.instances.runtime |
Service Account User role
On the default or user-managed service account in the project whereDataproc clusters are launched when you run pipelines, grant theCloud Data Fusion Service Account the Service Account User role(roles/iam.serviceAccountUser).
For more information, seeGrant service account permission.
Dataproc Worker role
To run the jobs on Dataproc clusters, grant the Dataproc Workerrole (roles/dataproc.worker) to the default or user-managed serviceaccounts used by your Cloud Data Fusion pipelines.
Roles for users
To trigger any operation in Cloud Data Fusion, you (the principal) musthave enough permissions. Individual permissions are grouped into roles, and yougrant roles to that principal.
Note: The rest of this page describes managing permissions using IAM for access control. To use RBAC, seeRBAC roles and permissions.If RBAC isn't enabled, or if you're using a Cloud Data Fusion edition thatdoesn't support RBAC, users with any Cloud Data Fusion IAMrole have full access to the Cloud Data Fusion webinterface. The Admin role only allows users to manage the instance, such asCreate,Update,Upgrade, andDelete operations.
Grant the following roles to principals, depending on the permissions theyneed in Cloud Data Fusion.
| Role | Description | Permissions |
|---|---|---|
Cloud Data Fusion Admin (roles/datafusion.admin) | All viewer permissions, plus permissions to create, update, and delete Cloud Data Fusion instances. |
|
Cloud Data Fusion Viewer (roles/datafusion.viewer) |
|
|
Access resources in another project at design time
This section describes access control on resources that are located in adifferent Google Cloud project than your Cloud Data Fusion instance at designtime.
When you design pipelines in the Cloud Data Fusion webinterface, you might use functions, such as Wrangler or Preview, whichaccess resources in other projects.
The following sections describe how you determine the service account in yourenvironment and then give the appropriate permissions.
Determine the service account of your environment
The service account name is Cloud Data Fusion Service Account and the principalfor this service agent isservice-CUSTOMER_PROJECT_NUMBER@gcp-sa-datafusion.iam.gserviceaccount.com.
Give permission to access resources in another project
To grant the roles that give permission to access various resources, followthese steps:
- In the project where the target resource exists, add theCloud Data Fusion Service Account (
service-CUSTOMER_PROJECT_NUMBER@gcp-sa-datafusion.iam.gserviceaccount.com) as a principal. - Grant roles to the Cloud Data Fusion Service Account on the targetresource in the project where the target resource exists.
After you grant the roles, you can access resources in a different project atdesign time in the same way that you access resources in the project where yourinstance is located.
Access resources in another project at execution time
This section describes access control on resources that are located in adifferent Google Cloud project than your Cloud Data Fusion instance atexecution time.
At execution time, you execute the pipeline on a Dataproccluster, which may access resources in other projects. By default, theDataproc cluster itself is launched in the same project asthe Cloud Data Fusion instance, but you can use clusters in anotherproject.
To access the resources in other Google Cloud projects, follow thesesteps:
- Determine the service account for your project.
- In the project where the resource is, grant IAM rolesto the Compute Engine default service account to give it access toresources in another project.
Determine the Compute Engine service account
For more information about the Compute Engine service account, seeAbout IAM in Cloud Data Fusion.
Grant IAM access resources in another project
The Compute Engine default service account requires permissions toaccess resources in another project. These roles and permissions can bedifferent depending on the resource you want to access.
To access the resources, follow these steps:
- Grant roles and permissions, specifying your Compute Engine serviceaccount as a principal in the project where the target resource exists.
- Add appropriate roles to access the resource.
Cloud Data Fusion API permissions
The following permissions are required to execute theCloud Data Fusion API.
| API call | Permission |
|---|---|
instances.create | datafusion.instances.create |
instances.delete | datafusion.instances.delete |
instances.list | datafusion.instances.list |
instances.get | datafusion.instances.get |
instances.update | datafusion.instances.update |
operations.cancel | datafusion.operations.cancel |
operations.list | datafusion.operations.list |
operations.get | datafusion.operations.get |
Permissions for common tasks
Common tasks in Cloud Data Fusion require the following permissions:
| Task | Permissions |
|---|---|
| Accessing the Cloud Data Fusion web interface | datafusion.instances.get |
| Accessing the Cloud Data FusionInstances page in the Google Cloud console | datafusion.instances.list |
| Accessing theDetails page for an instance | datafusion.instances.get |
| Creating a new instance | datafusion.instances.create |
| Updating labels and advanced options to customize an instance | datafusion.instances.update |
| Deleting an instance | datafusion.instances.delete |
What's next
- Learn more aboutaccess control between multiple projects.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-15 UTC.