BigQuery Data Transfer API Connector Overview
The Workflows connector defines the built-infunctions that can be used to access other Google Cloud products within aworkflow.
This page provides an overview of the individual connector.There is no need to import or load connector libraries in a workflow—connectorswork out of the box when used in a call step.
BigQuery Data Transfer API
Schedule queries or transfer external data from SaaS applications to Google BigQuery on a regular basis.To learn more, see theBigQuery Data Transfer API documentation.
BigQuery Data Transfer connector sample
YAML
# This workflow creates a new dataset and a new table inside that dataset, which are required# for the BigQuery Data Transfer Job to run. It creates a new TransferJob configuration and starts# a manual run of the transfer (30 seconds after the config is created).# The transferRun is a blocking LRO.# All resources get deleted once the transfer run completes.## On success, it returns "SUCCESS".## Features included in this test:# - BigQuery Data Transfer connector# - Waiting for long-running transfer run to complete## This workflow expects following items to be provided through input argument for execution:# - projectID (string)# - The user project ID.# - datasetID (string)# - The dataset name, expected to have an unique value to avoid the# instance being referred by multiple tests.# - tableID (string)# - The table name, expected to have an unique value to avoid the# instance being referred by multiple tests.# - runConfigDisplayName (string)# - The transfer run configuration display name.## Expected successful output: "SUCCESS"main:params:[args]steps:-init:assign:-project_id:${args.projectID}-destination_dataset:${args.datasetID}-destination_table:${args.tableID}-run_config_display_name:${args.runConfigDisplayName}-run_config_data_source_id:"google_cloud_storage"-location:"us"-data_path_template:"gs://xxxxxx-bucket/xxxxx/xxxx"-create_dataset:call:googleapis.bigquery.v2.datasets.insertargs:projectId:${project_id}body:datasetReference:datasetId:${destination_dataset}projectId:${project_id}-create_table:call:googleapis.bigquery.v2.tables.insertargs:datasetId:${destination_dataset}projectId:${project_id}body:tableReference:datasetId:${destination_dataset}projectId:${project_id}tableId:${destination_table}schema:fields:-name:"column1"type:"STRING"-name:"column2"type:"STRING"-list_config:call:googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.listargs:parent:${"projects/" + project_id + "/locations/us"}-create_run_config:call:googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.createargs:parent:${"projects/" + project_id + "/locations/" + location}body:displayName:${run_config_display_name}schedule:"everyday19:22"scheduleOptions:disableAutoScheduling:truedestinationDatasetId:${destination_dataset}dataSourceId:${run_config_data_source_id}params:destination_table_name_template:${destination_table}file_format:"CSV"data_path_template:${data_path_template}result:config-get_time_in_30s:assign:-now_plus_30s:${time.format(sys.now() + 30)}-start_run:call:googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.startManualRunsargs:parent:${config.name}body:requestedRunTime:${now_plus_30s}result:runsResp-remove_run_config:call:googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.deleteargs:name:${config.name}-delete_table:call:googleapis.bigquery.v2.tables.deleteargs:datasetId:${destination_dataset}projectId:${project_id}tableId:${destination_table}-delete_dataset:call:googleapis.bigquery.v2.datasets.deleteargs:projectId:${project_id}datasetId:${destination_dataset}-the_end:return:"SUCCESS"JSON
{"main":{"params":["args"],"steps":[{"init":{"assign":[{"project_id":"${args.projectID}"},{"destination_dataset":"${args.datasetID}"},{"destination_table":"${args.tableID}"},{"run_config_display_name":"${args.runConfigDisplayName}"},{"run_config_data_source_id":"google_cloud_storage"},{"location":"us"},{"data_path_template":"gs://xxxxxx-bucket/xxxxx/xxxx"}]}},{"create_dataset":{"call":"googleapis.bigquery.v2.datasets.insert","args":{"projectId":"${project_id}","body":{"datasetReference":{"datasetId":"${destination_dataset}","projectId":"${project_id}"}}}}},{"create_table":{"call":"googleapis.bigquery.v2.tables.insert","args":{"datasetId":"${destination_dataset}","projectId":"${project_id}","body":{"tableReference":{"datasetId":"${destination_dataset}","projectId":"${project_id}","tableId":"${destination_table}"},"schema":{"fields":[{"name":"column1","type":"STRING"},{"name":"column2","type":"STRING"}]}}}}},{"list_config":{"call":"googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.list","args":{"parent":"${\"projects/\" + project_id + \"/locations/us\"}"}}},{"create_run_config":{"call":"googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.create","args":{"parent":"${\"projects/\" + project_id + \"/locations/\" + location}","body":{"displayName":"${run_config_display_name}","schedule":"every day 19:22","scheduleOptions":{"disableAutoScheduling":true},"destinationDatasetId":"${destination_dataset}","dataSourceId":"${run_config_data_source_id}","params":{"destination_table_name_template":"${destination_table}","file_format":"CSV","data_path_template":"${data_path_template}"}}},"result":"config"}},{"get_time_in_30s":{"assign":[{"now_plus_30s":"${time.format(sys.now() + 30)}"}]}},{"start_run":{"call":"googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.startManualRuns","args":{"parent":"${config.name}","body":{"requestedRunTime":"${now_plus_30s}"}},"result":"runsResp"}},{"remove_run_config":{"call":"googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.delete","args":{"name":"${config.name}"}}},{"delete_table":{"call":"googleapis.bigquery.v2.tables.delete","args":{"datasetId":"${destination_dataset}","projectId":"${project_id}","tableId":"${destination_table}"}}},{"delete_dataset":{"call":"googleapis.bigquery.v2.datasets.delete","args":{"projectId":"${project_id}","datasetId":"${destination_dataset}"}}},{"the_end":{"return":"SUCCESS"}}]}}Module: googleapis.bigquerydatatransfer.v1.projects.dataSources
| Functions | |
|---|---|
checkValidCreds | Returns true if valid credentials exist for the given data source andrequesting user. |
get | Retrieves a supported data source and returns its settings. |
list | Lists supported data sources and returns their settings. |
Module: googleapis.bigquerydatatransfer.v1.projects.locations
| Functions | |
|---|---|
get | Gets information about a location. |
list | Lists information about the supported locations for this service. |
Module: googleapis.bigquerydatatransfer.v1.projects.locations.dataSources
| Functions | |
|---|---|
checkValidCreds | Returns true if valid credentials exist for the given data source andrequesting user. |
get | Retrieves a supported data source and returns its settings. |
list | Lists supported data sources and returns their settings. |
Module: googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs
| Functions | |
|---|---|
create | Creates a new data transfer configuration. |
delete | Deletes a data transfer configuration, including any associated transferruns and logs. |
get | Returns information about a data transfer config. |
list | Returns information about all transfer configs owned by a project in thespecified location. |
patch | Updates a data transfer configuration. All fields must be set, even ifthey are not updated. |
startManualRuns | Start manual transfer runs to be executed now with schedule_time equalto current time. The transfer runs can be created for a time range wherethe run_time is between start_time (inclusive) and end_time (exclusive),or for a specific run_time. |
Module: googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.runs
| Functions | |
|---|---|
delete | Deletes the specified transfer run. |
get | Returns information about the particular transfer run. |
list | Returns information about running and completed transfer runs. |
Module: googleapis.bigquerydatatransfer.v1.projects.locations.transferConfigs.runs.transferLogs
| Functions | |
|---|---|
list | Returns log messages for the transfer run. |
Module: googleapis.bigquerydatatransfer.v1.projects.transferConfigs
| Functions | |
|---|---|
create | Creates a new data transfer configuration. |
delete | Deletes a data transfer configuration, including any associated transferruns and logs. |
get | Returns information about a data transfer config. |
list | Returns information about all transfer configs owned by a project in thespecified location. |
patch | Updates a data transfer configuration. All fields must be set, even ifthey are not updated. |
startManualRuns | Start manual transfer runs to be executed now with schedule_time equalto current time. The transfer runs can be created for a time range wherethe run_time is between start_time (inclusive) and end_time (exclusive),or for a specific run_time. |
Module: googleapis.bigquerydatatransfer.v1.projects.transferConfigs.runs
| Functions | |
|---|---|
delete | Deletes the specified transfer run. |
get | Returns information about the particular transfer run. |
list | Returns information about running and completed transfer runs. |
Module: googleapis.bigquerydatatransfer.v1.projects.transferConfigs.runs.transferLogs
| Functions | |
|---|---|
list | Returns log messages for the transfer run. |
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2026-02-19 UTC.