Control access with IAM

This document shows you how to do the following in Dataform:

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.
    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    Go to project selector

  3. Verify that billing is enabled for your Google Cloud project.

  4. Enable the BigQuery and Dataform APIs.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

    Enable the APIs

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.
    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    Go to project selector

  6. Verify that billing is enabled for your Google Cloud project.

  7. Enable the BigQuery and Dataform APIs.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

    Enable the APIs

Grant Dataform the required access

This section shows you how to grant the Identity and Access Management (IAM) rolesthat Dataform service agents and custom service accountsrequire to run workflows in BigQuery.

About custom service accounts and Dataform service agents

You can configure custom service accounts to run workflows on your behalf in thefollowing ways:

When you create a Dataform repository or workflow configuration,you can select any service account that you haveact-as permissionson. You must configure the required permissions for all theservice accounts associated with your Dataform resources.

Important: We recommend enablingstrict act-as mode,as this helps to ensure a more secure and predictable permissions model for your Dataform projects.

When you create your first Dataform repository,Dataform automatically generates a default service agent.Dataform uses the default service agent to interact withBigQuery on your behalf.

Your default Dataform service agent ID is in the followingformat:

service-PROJECT_NUMBER@gcp-sa-dataform.iam.gserviceaccount.com

ReplacePROJECT_NUMBER with the numeral ID of yourGoogle Cloud project. You can find your Google Cloud project ID in theGoogle Cloud console dashboard. For more information, seeIdentifying projects.

Required roles for Dataform service agents, custom service accounts, and Google Accounts

Default Dataform service agents, custom service accounts, andGoogle Account user credentials(Preview) used toauthenticate in Dataform require the followingBigQuery IAM roles to be able to run workflows inBigQuery:

Additionally, grant the following roles to the default Dataformservice agent on theeffective service accountfor the workflow configuration:

Caution: When you grant the Service Account Token Creator role to a defaultDataform service agent so that it can access a custom serviceaccount in your project, the custom service account can then be used in anyDataform repository. This means that anyone who can runworkflows in Dataform can use this service account in theproject. To mitigate this risk, we recommend enablingstrict act-as mode.

For automatic repository releases and automatic workflow runs,grant the default Dataform service agent theiam.serviceAccounts.actAs permission on theeffective service account.

Security considerations

Granting the roles required by Dataform to aDataform service agent, custom service account, or auser's Google Account (Preview)comes with the following security considerations:

  • Any service agent or service account granted the required roles might gainaccess to BigQuery or Secret Manager in the project thatthe service agent or service account belongs to, regardless ofVPC Service Controls.Requests originating from Dataform that use a service agent with the required roles are within the VPC Service Controls perimeter of the project that the Dataform repository belongs to.

    For more information, seeConfigure VPC Service Controls.

  • Any user who has thedataform.repositories.create IAMpermission can run code using the default Dataform serviceagent and all the permissions granted to that service agent or serviceaccount.

    For more information, seeSecurity considerations for Dataform permissions.

To restrict the data that a user, service agent, or service account canread or write in BigQuery, you can grant granularBigQuery IAM permissions to selectedBigQuery datasets or tables. For more information, seeControlling access to datasets andControlling access to tables and views.

To prevent users from performing actions while using the Google Account usercredentials of another user, the following restrictions are enforced:

  • To modify a workflow configuration with another Google Account user'scredentials attached to it, you need to attach your own Google Accountuser credentials to the workflow configuration or change the workflowconfiguration to authenticate with a {dataform_name_short} service agentor custom service account.
  • You can't modify a compilation result for a release configuration if thereare workflow configurations referencing the release configuration that haveanother Google Account user's credentials attached.
  • You can't set a workflow configuration to authenticate with Google Accountuser credentials and reference a release configuration with a schedule. Thislimitation has the following consequences:

    • You can't update a release configuration to use a schedule if there areworkflow configurations referencing the release configuration that are setto authenticate with Google Account user credentials.
    • You can't create a workflow configuration that authenticates withGoogle Account user credentials and points to a release configuration with aschedule.
    • You can't create or update a workflow configuration to use Google Accountuser credentials and point to a release configuration with a schedule.

Grant the required BigQuery roles

To grant the required BigQuery IAM roles to yourdefault Dataform service agent, a custom service accountthat you want to use in Dataform, or a user's Google Account that youwant to use to authenticate in Dataform(Preview), follow these steps:

  1. In the Google Cloud console, go to theDataform page.

    Go to Dataform

  2. Select orcreate a repository.

  3. In the Google Cloud console, go to theIAM page.

    Go to IAM

  4. ClickGrant Access.

  5. In theNew principals field, enter the service agent ID, serviceaccount ID, or the user's GoogleAccount email (Preview).

  6. In theSelect a role list, select theBigQuery Job User role.

  7. ClickAdd another role, and then in theSelect a role list,select theBigQuery Data Editor role.

  8. ClickAdd another role, and then in theSelect a role list,select theBigQuery Data Viewer role.

  9. ClickSave.

Grant roles required for automatic workflows

To use a custom service account in Dataform, the defaultDataform service agent must be able to access the custom serviceaccount. This lets Dataform run your workflows using thepermissions defined on your custom service account instead of on the defaultservice agent's account.

To grant this access, you need to grant theService Account Token Creator role(roles/iam.serviceAccountTokenCreator) to the default Dataformservice agent as the principal. This lets the default Dataformservice agent impersonate the service account by creating short-lived credentialsknown as tokens. These tokens are required for Dataform to runworkflows using the custom service account's identity.

You also need to grant theService Account User role(roles/iam.serviceAccountUser) to the default Dataform serviceagent. This lets the default Dataform service agent start newautomatic workflow runs for workflow configurations that are run by the custom service account.

To grant the default Dataform service agent access to acustom service account, follow these steps:

  1. In the Google Cloud console, go toIAM>Service accounts.

    Go to Service accounts

  2. Select a project.

  3. On theService accounts for project "PROJECT_NAME"page, select your custom service account.

  4. Go toPrincipals with access, and then clickGrant Access.

  5. In theNew principals field, enter your default Dataformservice agent ID.

    Your default Dataform service agent ID is in the followingformat:

    service-PROJECT_NUMBER@gcp-sa-dataform.iam.gserviceaccount.com
  6. In theSelect a role list, select theService Account Token Creatorrole and theService Account User role.

  7. ClickSave.

The custom service account is now ready to be configured within yourDataform repository.

Audit service account configurations

This section shows you how to audit your Dataform resources toensure proper service account usage and permission grants. Auditing is especiallyimportant when using custom service accounts, as they require specificpermissions for the default Dataform service agent to operate.

When using a custom service account for a Dataform repository orworkflow configuration, you must verify that the defaultDataform service agent has theService Account User role(roles/iam.serviceAccountUser) on the custom service account. This rolegrants theiam.serviceAccounts.actAs permission, which lets scheduled runs,initiated by the default Dataform service agent, impersonatethe custom service account. Additionally, verify that the defaultDataform service agent has theService Account Token Creator role(roles/iam.serviceAccountTokenCreator)on the effective service account.

Verify repository service accounts

First, identify thedataform.Repository assets that are in scope forDataform's own scheduling and execution. Then, verify the serviceaccount configurations for those in-scope repositories.

  1. Use Cloud Asset Inventory to list all the resources of thedataform.Repository type.For more information, seeView your assets.

  2. For each repository in the Cloud Asset Inventory output, check theresource.data.labels field to determine if it's in scope. The exact path mightvary slightly based on your export format.

  3. Identify out-of-scope repositories by inspecting the labels map for thesingle-file-asset-type key. The presence of this key indicates that therepository is used by a BigQuery feature. If the value issqlordata_canvas, the repository can be excluded from the service accountpermission checks.

    The remaining repositories lacking this key or these values are in scope for the service account permission checks.

  4. For each in-scope repository, check theresource.data.serviceAccount fieldin the Cloud Asset Inventory output to determine if a custom service account isconfigured:

  5. If a custom service account is used, verify that the defaultDataform service agent has both the Service Account User role(roles/iam.serviceAccountUser) and the Service Account Token Creator role(roles/iam.serviceAccountTokenCreator) on that custom service account.

Verify workflow configuration service accounts

Using dedicated custom service accounts for Dataform workflowconfigurations is a security best practice, aligning with the principle of leastprivilege.

To verify service account usage fordataform.WorkflowConfig resources, dothe following:

  1. Use Cloud Asset Inventory to list all resources of thedataform.WorkflowConfigtype.

  2. For each workflow configuration, examine the Cloud Asset Inventory output to determine the effective service account:

    • If theresource.data.serviceAccount field is present, this value isthe email address of the service account explicitly set on the workflowconfiguration.
    • If theresource.data.serviceAccount field is absent, the workflowconfiguration inherits the service account from its parent repository.Check theparent repository's configurationto find the effective service account.
  3. Identify if a custom service account is being used by comparing the emailaddress of the effective service account with the email address of the project'sdefault Dataform service agent. If they are different, a customservice account is in use.

  4. If a custom service account is in use, ensure that the defaultDataform service agent has both the Service Account User role(roles/iam.serviceAccountUser) and the Service Account Token Creator role(roles/iam.serviceAccountTokenCreator) granted on that custom serviceaccount. These permissions let the default Dataform serviceagent initiate workflow executions impersonating the custom service account.

Control access to Dataform with IAM

This section describes the access control options for Dataformand shows you how to view and grant Dataform roles.Dataform usesIdentity and Access Management (IAM)for access control. For more information about roles and permissions inIAM, seeIAM roles and permissions index.

Predefined Dataform roles

The following table lists the predefined roles that give you accessto Dataform resources:

RolePermissions

Dataform Admin

(roles/dataform.admin)

Full access to all Dataform resources.

dataform.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update
  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update
  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query
  • dataform.config.get
  • dataform.config.update
  • dataform.folders.addContents
  • dataform.folders.create
  • dataform.folders.delete
  • dataform.folders.get
  • dataform.folders.getIamPolicy
  • dataform.folders.move
  • dataform.folders.queryContents
  • dataform.folders.setIamPolicy
  • dataform.folders.update
  • dataform.locations.get
  • dataform.locations.list
  • dataform.operations.cancel
  • dataform.operations.delete
  • dataform.operations.get
  • dataform.operations.list
  • dataform.releaseConfigs.create
  • dataform.releaseConfigs.delete
  • dataform.releaseConfigs.get
  • dataform.releaseConfigs.list
  • dataform.releaseConfigs.update
  • dataform.repositories.commit
  • dataform.repositories.computeAccessTokenStatus
  • dataform.repositories.create
  • dataform.repositories.delete
  • dataform.repositories.fetchHistory
  • dataform.repositories.fetchRemoteBranches
  • dataform.repositories.get
  • dataform.repositories.getIamPolicy
  • dataform.repositories.list
  • dataform.repositories.move
  • dataform.repositories.queryDirectoryContents
  • dataform.repositories.readFile
  • dataform.repositories.scheduleRelease
  • dataform.repositories.scheduleWorkflow
  • dataform.repositories.setIamPolicy
  • dataform.repositories.update
  • dataform.teamFolders.create
  • dataform.teamFolders.delete
  • dataform.teamFolders.get
  • dataform.teamFolders.getIamPolicy
  • dataform.teamFolders.setIamPolicy
  • dataform.teamFolders.update
  • dataform.workflowConfigs.create
  • dataform.workflowConfigs.delete
  • dataform.workflowConfigs.get
  • dataform.workflowConfigs.list
  • dataform.workflowConfigs.update
  • dataform.workflowInvocations.cancel
  • dataform.workflowInvocations.create
  • dataform.workflowInvocations.delete
  • dataform.workflowInvocations.get
  • dataform.workflowInvocations.list
  • dataform.workflowInvocations.query
  • dataform.workspaces.commit
  • dataform.workspaces.create
  • dataform.workspaces.delete
  • dataform.workspaces.fetchFileDiff
  • dataform.workspaces.fetchFileGitStatuses
  • dataform.workspaces.fetchGitAheadBehind
  • dataform.workspaces.get
  • dataform.workspaces.getIamPolicy
  • dataform.workspaces.installNpmPackages
  • dataform.workspaces.list
  • dataform.workspaces.makeDirectory
  • dataform.workspaces.moveDirectory
  • dataform.workspaces.moveFile
  • dataform.workspaces.pull
  • dataform.workspaces.push
  • dataform.workspaces.queryDirectoryContents
  • dataform.workspaces.readFile
  • dataform.workspaces.removeDirectory
  • dataform.workspaces.removeFile
  • dataform.workspaces.reset
  • dataform.workspaces.searchFiles
  • dataform.workspaces.setIamPolicy
  • dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Code CommenterBeta

(roles/dataform.codeCommenter)

Permissions to comment, at the repository level. Grants CRUD access over commentThread and comment resources.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.folders.get

dataform.folders.queryContents

dataform.locations.get

dataform.repositories.get

dataform.repositories.readFile

resourcemanager.projects.get

resourcemanager.projects.list

Code Creator

(roles/dataform.codeCreator)

Access only to private and shared code resources. The permissions in the Code Creator let you create and list code in Dataform, and access only the code that you created and code that was explicitly shared with you.

dataform.commentThreads.get

dataform.commentThreads.list

dataform.comments.get

dataform.comments.list

dataform.folders.create

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.repositories.create

dataform.repositories.list

resourcemanager.projects.get

resourcemanager.projects.list

Code Editor

(roles/dataform.codeEditor)

Edit access code resources.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.addContents

dataform.folders.create

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.repositories.commit

dataform.repositories.computeAccessTokenStatus

dataform.repositories.create

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.workspaces.commit

dataform.workspaces.create

dataform.workspaces.delete

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.installNpmPackages

dataform.workspaces.list

dataform.workspaces.makeDirectory

dataform.workspaces.moveDirectory

dataform.workspaces.moveFile

dataform.workspaces.pull

dataform.workspaces.push

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.removeDirectory

dataform.workspaces.removeFile

dataform.workspaces.reset

dataform.workspaces.searchFiles

dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Code Owner

(roles/dataform.codeOwner)

Full access to code resources.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.*

  • dataform.folders.addContents
  • dataform.folders.create
  • dataform.folders.delete
  • dataform.folders.get
  • dataform.folders.getIamPolicy
  • dataform.folders.move
  • dataform.folders.queryContents
  • dataform.folders.setIamPolicy
  • dataform.folders.update

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.operations.list

dataform.repositories.commit

dataform.repositories.computeAccessTokenStatus

dataform.repositories.create

dataform.repositories.delete

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.move

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.repositories.setIamPolicy

dataform.repositories.update

dataform.workspaces.*

  • dataform.workspaces.commit
  • dataform.workspaces.create
  • dataform.workspaces.delete
  • dataform.workspaces.fetchFileDiff
  • dataform.workspaces.fetchFileGitStatuses
  • dataform.workspaces.fetchGitAheadBehind
  • dataform.workspaces.get
  • dataform.workspaces.getIamPolicy
  • dataform.workspaces.installNpmPackages
  • dataform.workspaces.list
  • dataform.workspaces.makeDirectory
  • dataform.workspaces.moveDirectory
  • dataform.workspaces.moveFile
  • dataform.workspaces.pull
  • dataform.workspaces.push
  • dataform.workspaces.queryDirectoryContents
  • dataform.workspaces.readFile
  • dataform.workspaces.removeDirectory
  • dataform.workspaces.removeFile
  • dataform.workspaces.reset
  • dataform.workspaces.searchFiles
  • dataform.workspaces.setIamPolicy
  • dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Code SchedulerBeta

(roles/dataform.codeScheduler)

Access for scheduling workflows and releases.

dataform.releaseConfigs.create

dataform.workflowConfigs.create

Code Viewer

(roles/dataform.codeViewer)

Read-only access to all code resources.

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.repositories.computeAccessTokenStatus

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.list

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.searchFiles

resourcemanager.projects.get

resourcemanager.projects.list

Dataform Editor

(roles/dataform.editor)

Edit access to Workspaces and Read-only access to Repositories.

dataform.commentThreads.get

dataform.commentThreads.list

dataform.comments.get

dataform.comments.list

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.config.get

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.operations.list

dataform.releaseConfigs.get

dataform.releaseConfigs.list

dataform.repositories.computeAccessTokenStatus

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.workflowConfigs.get

dataform.workflowConfigs.list

dataform.workflowInvocations.*

  • dataform.workflowInvocations.cancel
  • dataform.workflowInvocations.create
  • dataform.workflowInvocations.delete
  • dataform.workflowInvocations.get
  • dataform.workflowInvocations.list
  • dataform.workflowInvocations.query

dataform.workspaces.commit

dataform.workspaces.create

dataform.workspaces.delete

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.installNpmPackages

dataform.workspaces.list

dataform.workspaces.makeDirectory

dataform.workspaces.moveDirectory

dataform.workspaces.moveFile

dataform.workspaces.pull

dataform.workspaces.push

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.removeDirectory

dataform.workspaces.removeFile

dataform.workspaces.reset

dataform.workspaces.searchFiles

dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Dataform Service Agent

(roles/dataform.serviceAgent)

Gives permission for the Dataform API to access a secret from Secret Manager

Warning: Do not grant service agent roles to any principals exceptservice agents.

dataform.compilationResults.create

dataform.workflowInvocations.create

resourcemanager.projects.get

resourcemanager.projects.list

Team Folder CommenterBeta

(roles/dataform.teamFolderCommenter)

View and comment access to a team folder and its contents.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.repositories.computeAccessTokenStatus

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.list

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.searchFiles

resourcemanager.projects.get

resourcemanager.projects.list

Team Folder ContributorBeta

(roles/dataform.teamFolderContributor)

Edit access to a team folder and its contents.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.addContents

dataform.folders.create

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.repositories.commit

dataform.repositories.computeAccessTokenStatus

dataform.repositories.create

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.teamFolders.update

dataform.workspaces.commit

dataform.workspaces.create

dataform.workspaces.delete

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.installNpmPackages

dataform.workspaces.list

dataform.workspaces.makeDirectory

dataform.workspaces.moveDirectory

dataform.workspaces.moveFile

dataform.workspaces.pull

dataform.workspaces.push

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.removeDirectory

dataform.workspaces.removeFile

dataform.workspaces.reset

dataform.workspaces.searchFiles

dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Team Folder CreatorBeta

(roles/dataform.teamFolderCreator)

Access to create new team folders.

dataform.teamFolders.create

Team Folder OwnerBeta

(roles/dataform.teamFolderOwner)

Full access to a team folder and its contents. Can share the team folder and its contents.

dataform.commentThreads.*

  • dataform.commentThreads.create
  • dataform.commentThreads.delete
  • dataform.commentThreads.get
  • dataform.commentThreads.list
  • dataform.commentThreads.update

dataform.comments.*

  • dataform.comments.create
  • dataform.comments.delete
  • dataform.comments.get
  • dataform.comments.list
  • dataform.comments.update

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.*

  • dataform.folders.addContents
  • dataform.folders.create
  • dataform.folders.delete
  • dataform.folders.get
  • dataform.folders.getIamPolicy
  • dataform.folders.move
  • dataform.folders.queryContents
  • dataform.folders.setIamPolicy
  • dataform.folders.update

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.operations.list

dataform.repositories.commit

dataform.repositories.computeAccessTokenStatus

dataform.repositories.create

dataform.repositories.delete

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.move

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.repositories.setIamPolicy

dataform.repositories.update

dataform.teamFolders.delete

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.teamFolders.setIamPolicy

dataform.teamFolders.update

dataform.workspaces.*

  • dataform.workspaces.commit
  • dataform.workspaces.create
  • dataform.workspaces.delete
  • dataform.workspaces.fetchFileDiff
  • dataform.workspaces.fetchFileGitStatuses
  • dataform.workspaces.fetchGitAheadBehind
  • dataform.workspaces.get
  • dataform.workspaces.getIamPolicy
  • dataform.workspaces.installNpmPackages
  • dataform.workspaces.list
  • dataform.workspaces.makeDirectory
  • dataform.workspaces.moveDirectory
  • dataform.workspaces.moveFile
  • dataform.workspaces.pull
  • dataform.workspaces.push
  • dataform.workspaces.queryDirectoryContents
  • dataform.workspaces.readFile
  • dataform.workspaces.removeDirectory
  • dataform.workspaces.removeFile
  • dataform.workspaces.reset
  • dataform.workspaces.searchFiles
  • dataform.workspaces.setIamPolicy
  • dataform.workspaces.writeFile

resourcemanager.projects.get

resourcemanager.projects.list

Team Folder ViewerBeta

(roles/dataform.teamFolderViewer)

View access to a team folder and its contents.

dataform.compilationResults.*

  • dataform.compilationResults.create
  • dataform.compilationResults.get
  • dataform.compilationResults.list
  • dataform.compilationResults.query

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.repositories.computeAccessTokenStatus

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.list

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.searchFiles

resourcemanager.projects.get

resourcemanager.projects.list

Dataform Viewer

(roles/dataform.viewer)

Read-only access to all Dataform resources.

dataform.commentThreads.get

dataform.commentThreads.list

dataform.comments.get

dataform.comments.list

dataform.compilationResults.get

dataform.compilationResults.list

dataform.compilationResults.query

dataform.config.get

dataform.folders.get

dataform.folders.getIamPolicy

dataform.folders.queryContents

dataform.locations.*

  • dataform.locations.get
  • dataform.locations.list

dataform.operations.get

dataform.operations.list

dataform.releaseConfigs.get

dataform.releaseConfigs.list

dataform.repositories.computeAccessTokenStatus

dataform.repositories.fetchHistory

dataform.repositories.fetchRemoteBranches

dataform.repositories.get

dataform.repositories.getIamPolicy

dataform.repositories.list

dataform.repositories.queryDirectoryContents

dataform.repositories.readFile

dataform.teamFolders.get

dataform.teamFolders.getIamPolicy

dataform.workflowConfigs.get

dataform.workflowConfigs.list

dataform.workflowInvocations.get

dataform.workflowInvocations.list

dataform.workflowInvocations.query

dataform.workspaces.fetchFileDiff

dataform.workspaces.fetchFileGitStatuses

dataform.workspaces.fetchGitAheadBehind

dataform.workspaces.get

dataform.workspaces.getIamPolicy

dataform.workspaces.list

dataform.workspaces.queryDirectoryContents

dataform.workspaces.readFile

dataform.workspaces.searchFiles

resourcemanager.projects.get

resourcemanager.projects.list

Custom Dataform roles

Custom roles can include any permissions that you specify. You can createcustom roles that include permissions to perform specific administrativeoperations, like creating development workspaces or creating files anddirectories within a development workspace. To create custom roles,seeCreating and managing custom roles.

Security considerations for Dataform permissions

Any user who has thedataform.repositories.create permission can run codein BigQuery using the default Dataform serviceagent and all permissions granted to that service agent. This includesexecution of Dataformworkflows.

Thedataform.repositories.create permissions is included in the followingIAM roles:

To restrict the data that a user, service agent, or service account canread or write in BigQuery, you can grant granularBigQuery IAM permissions to selectedBigQuery datasets or tables. For more information, seeControlling access to datasets andControlling access to tables and views.

For more information about the default Dataform service agentand the roles and permissions it requires, seeGrant Dataform required access.

View Dataform roles

Within the Google Cloud console, perform the following steps:

  1. Go to theIAM & Admin> Roles page.

    Go to Roles

  2. In theFilter field, selectUsed in, typeDataform,and then pressEnter.

  3. Click one of the listed roles to view the permissions of the role in theright pane.

    For example, the Dataform Admin role has full accessto all Dataform resources.

For more information about granting a role on a project, seeGrant a role.You can grant predefined or custom roles in this way.

Control access to an individual repository

To control access to Dataform with granular permissions,you can set Dataform IAM roles on individualrepositories by using the Dataform APIrepositories.setIamPolicyrequest.

To set Dataform IAM roles on an individualDataform repository, follow these steps:

  1. In the terminal, pass the Dataform APIrepositories.setIamPolicyrequest with an accesspolicy.

  2. In the policy, bind a user, group, domain, service agent, or serviceaccount to a selected role in the following format:

    {"policy":   {      "bindings": [      {         "role": "roles/ROLE",         "members": [            "TYPE:IDENTIFIER",         ]      },      ],   }}

    Replace the following:

    • ROLE: the DataformIAM role that you want to grant on the repository.
    • TYPE:user,group,domain, orserviceAccount.
    • IDENTIFIER: the user, group, domain, or serviceaccount that you want to grant the role to.
  3. In the IAM page, ensure that all users can view the full listof Dataform repositories through aDataform role with thedataform.repositories.list permission.

  4. In IAM, ensure that only users who require full access to allDataform repositories are granted the Dataform Admin role on allrepositories.

The following command passes therepositories.setIamPolicy Dataform APIrequest that grants the Dataform Editor role on thesales repositoryto a single user:

curl -H "Content-Type: application/json" -X POST -d '{ "policy": { "bindings": [{ "role": "roles/dataform.editor", "members": ["user:sasha@examplepetstore.com"]}] }}' "https://dataform.googleapis.com/v1/projects/examplepetstore/locations/us-central1/repositories/sales:setIamPolicy"

Grant public access to a repository

You can grant public access to a Dataform repository by grantingIAM roles on the repository to theallAuthenticatedUsersprincipal.

When you assign an IAM role to theallAuthenticatedUsersprincipal, service agents, service accounts, and all users on the internetwho have authenticated with a Google Account are granted that role. Thisincludes accounts that aren't connected to a Google Workspace accountor Cloud Identity domain, such as personal Gmail accounts. Userswho aren't authenticated, such as anonymous visitors, aren't included. Formore information, seeAll authenticated users.

For example, when you grant the Dataform Viewer role toallAuthenticatedUsers on thesales repository, all service agents,service accounts, and users on the internet who have authenticated with aGoogle Account have read-only access to allsales code resources.

Caution: Granting administrator, edit, or execution level permissionstoallAuthenticatedUsers can allow bad actors to access yourdata and run code. Grant only the minimal required permissions.

To grant public access to a Dataform repository, follow thesesteps:

  1. In the terminal, pass the Dataform APIrepositories.setIamPolicyrequest with an accesspolicy.

  2. In the policy, bind theallAuthenticatedUsers principal to a selectedrole in the following format:

    {"policy":   {      "bindings": [      {         "role": "roles/ROLE",         "members": [            "allAuthenticatedUsers",         ]      },      ],   }}

    ReplaceROLE with a DataformIAM role that you want to grant to all authenticatedusers.

The following command passes therepositories.setIamPolicy Dataform APIrequest that grants the Dataform Viewer role on thesales repositorytoallAuthenticatedUsers:

curl -H "Content-Type: application/json" -X POST -d '{ "policy": { "bindings": [{ "role": "roles/dataform.viewer", "members": ["allAuthenticatedUsers"]}] }}' "https://dataform.googleapis.com/v1/projects/examplepetstore/locations/us-central1/repositories/sales:setIamPolicy"

Prevent public access to repositories

To ensure that no access is granted to the public on any Dataformrepository, you can restrict theallAuthenticatedUsers principal in yourproject.

To restrictallAuthenticatedUsers in your project, you canset theiam.allowedPolicyMemberDomains policy,and removeallAuthenticatedUsers from the list ofallowed_values.

When you restrictallAuthenticatedUsers in theiam.allowedPolicyMemberDomainspolicy, theallAuthenticatedUsers principal can't be used in anyIAM policy in your project, which prevents granting public accessto all resources, including Dataform repositories.

For more information about theiam.allowedPolicyMemberDomains policyand also instructions to set it, seeRestricting identities by domain.

Workforce Identity Federation in Dataform

Workforce Identity Federationlets you use an external identity provider (IdP) toauthenticate and authorize users to Google Cloud services withIAM.

Dataform supports Workforce Identity Federation withno known limitations.

Control access to individual tables with IAM

This section shows you how to grant and revoke BigQueryIAM rolesfor individual Dataform tables and views.

When Dataform runs a table or view, it creates the resourcein BigQuery. During development in Dataform, youcan grant BigQuery roles to individual tables and views tocontrol their access in BigQuery after execution.

For more information about granting and revoking access to resources, seeGrant access to a resource.

Grant BigQuery roles to a table or view

You can grant BigQuery roles to a table or view inDataform by adding apost_operations block with theGRANT DCL statementto the.sqlx definition file of the selected table or view.

To grant BigQuery roles to a selected table or view, follow thesesteps:

  1. In the Google Cloud console, go to theDataform page.

    Go to Dataform

  2. Select a repository, and then select a workspace.

  3. In theFiles pane, expand thedefinitions/ directory.

  4. Select the.sqlx definition file of the table or view that you want togrant access to.

  5. In the file, enter the following code snippet:

    post_operations{GRANT"ROLE_LIST"ON"RESOURCE_TYPE"${self()}TO"USER_LIST"}

    Replace the following:

    • ROLE_LIST: the BigQuery role or list ofcomma-separated BigQuery roles that you want to grant.

    • RESOURCE_TYPE:TABLE orVIEW.

    • USER_LIST: the comma-separated list of users that therole is granted to.

      For a list of valid formats, seeuser_list.

  6. Optional: ClickFormat.

  7. Run the table or view.

  8. If you granted access on an incremental table, remove theGRANT statementfrom the table definition file after the first execution.

The following code sample shows theBigQuery Viewer rolegranted on a table to a user:

config { type: "table" }SELECT ...post_operations {  GRANT `roles/bigquery.dataViewer`  ON TABLE ${self()}  TO "user:222larabrown@gmail.com"}

Revoke BigQuery roles from a table or view

You can revoke BigQuery roles from a table or view by adding apost_operations block with theREVOKE DCL statementto the.sqlx definition file of the selected table or view.

To revoke BigQuery roles from a selected table or view, followthese steps:

  1. In the Google Cloud console, go to theDataform page.

    Go to Dataform

  2. Select a repository, and then select a workspace.

  3. In theFiles pane, expand thedefinitions/ directory.

  4. Select the.sqlx definition file of the table or view that you want torevoke access to.

  5. In thepost_operations block, enter the followingREVOKE statement:

    REVOKE"ROLE_LIST"ON"RESOURCE_TYPE"${self()}FROM"USER_LIST"

    Replace the following:

    • ROLE_LIST: the BigQuery role or list ofcomma-separated BigQuery roles that you want to revoke.
    • RESOURCE_TYPE:TABLE orVIEW.
    • USER_LIST: the comma-separated list of users that therole is revoked from. For a list of valid formats, seeuser_list.
  6. To revoke the access granted in aGRANT statement in the file, replace theGRANT statement with aREVOKE statement.

    Warning: Removing theGRANT statement without adding theREVOKE statement does not revoke access.
  7. Optional: ClickFormat.

  8. Run the table or view.

  9. If you revoked access to an incremental table, remove theREVOKE statementfrom the table definition file after the first execution.

The following code sample shows theBigQuery Viewer rolerevoked from a user on a table:

config { type: "table" }SELECT ...post_operations {  REVOKE `roles/bigquery.dataViewer`  ON TABLE ${self()}  FROM "user:222larabrown@gmail.com"}

Collectively manage BigQuery roles for tables and views

To control BigQuery access to individual tables and views in asingle location, you can create a dedicatedtype: "operations" file withGRANT andREVOKEDCL statements.

To manage BigQuery table access in a singletype: "operations"file, follow these steps:

  1. In the Google Cloud console, go to theDataform page.

    Go to Dataform

  2. Select a repository, and then select a workspace.

  3. In theFiles pane, next todefinitions/, click theMore menu.

  4. ClickCreate file.

  5. In theAdd a file path field, enter the name of the file followed by.sqlx afterdefinitions/. For example,definitions/table-access.sqlx.

    Filenames can only include numbers, letters, hyphens, and underscores.

  6. ClickCreate file.

  7. In theFiles pane, expand thedefinitions/ directory, and select thenewly created file.

  8. In the file, enter the following code snippet:

    config{type:"operations"}GRANT"ROLE_LIST"ONRESOURCE_TYPERESOURCE_NAMETO"USER_LIST"REVOKE"ROLE_LIST"ON{"<var>"}}RESOURCE_TYPERESOURCE_NAMETO"USER_LIST"

    Replace the following:

    • ROLE_LIST: the BigQuery role or list ofcomma-separated BigQuery roles that you want to grant orrevoke.
    • RESOURCE_TYPE:TABLE orVIEW.
    • RESOURCE_NAME: the name of the table or view.
    • USER_LIST: the comma-separated list of users that therole is granted to or revoked from. For a list of valid formats, seeuser_list.
  9. AddGRANT andREVOKE statements as needed.

    1. To revoke access granted in aGRANT statement in the file, replace theGRANT statement with aREVOKE statement.

      Removing theGRANT statement without adding theREVOKE statementdoes not revoke access.

  10. Optional: ClickFormat.

  11. Run the file after each update.

    1. If you granted or revoked access on an incremental table, remove theGRANT orREVOKE statement from the file after the first execution ofthe statement.

Use theConfig API to customize IAM behavior

You can use theprojects.locations.updateConfig Dataform API methodto customize IAM behavior and enhance security.

Warning: You must call this method for each location where you want toenhance security. When the configuration is applied to a location, it affectsall the Dataform repositories in that location. To limit thelocations where Dataform resources can be created, seeRestricting resource locations.

Grant a specific role upon resource creation

When you set thesetAuthenticatedUserAdmin field totrue in theprojects.locations.repositories resource,Dataform automatically grants the user who creates therepository theDataform Admin role(roles/dataform.admin) on that repository. In addition,Dataform grants any user who creates a workspace in thatrepository the Dataform Admin role on the new workspace.

You can use thecreator_role field(Preview)in theprojects.locations.updateConfig methodto override this behavior. IfsetAuthenticatedUserAdmin istrue and youconfigure thecreator_role field with a custom role, Dataformgrants the custom role instead of the defaultdataform.admin role.

Implement enhanced scheduling permissions

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Note: To provide feedback or request support, contactdataform-preview-support@google.com.

To require that users need explicit permissions to scheduleDataform workflows, set theenable_project_checks_for_scheduling field totrue in theprojects.locations.updateConfig method.

When you enable these checks for scheduling, the user needs the followingpermissions:

  • To create a workflow configuration:

    • Thedataform.workflowConfigs.create permission on the project, grantedby theCode Scheduler role(roles/dataform.codeScheduler).
    • Thedataform.repositories.scheduleWorkflow permission on the repository,granted by theDataform Admin role(roles/dataform.admin).
  • To create a release configuration:

    • Thedataform.releaseConfigs.create permission on the project, grantedby theCode Scheduler role(roles/dataform.codeScheduler).
    • Thedataform.repositories.scheduleRelease permission on the repository,granted by theDataform Admin role(roles/dataform.admin).

Enable private workspaces

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Note: To provide feedback or request support, contactdataform-preview-support@google.com.

To restrict Dataform workspace access so that only the workspacecreator can read and write code in that workspace, set theenable_private_workspace field totrue in theprojects.locations.updateConfig method.

This restriction also applies to viewing generated artifacts, such as compiledSQL, compilation errors, and run logs for the workspace's compilations orworkflow invocations.

This setting overrides standard IAM roles that grant workspaceaccess to other repository users.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-16 UTC.