Create a transfer configuration with run notifications

Create a transfer configuration, where notifications are sent to Cloud Pub/Sub when a transfer run succeeds or fails.

Explore further

For detailed documentation that includes this code sample, see the following:

Code sample

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.api.gax.rpc.ApiException;importcom.google.cloud.bigquery.datatransfer.v1.CreateTransferConfigRequest;importcom.google.cloud.bigquery.datatransfer.v1.DataTransferServiceClient;importcom.google.cloud.bigquery.datatransfer.v1.ProjectName;importcom.google.cloud.bigquery.datatransfer.v1.TransferConfig;importcom.google.protobuf.Struct;importcom.google.protobuf.Value;importjava.io.IOException;importjava.util.HashMap;importjava.util.Map;// Sample to get run notificationpublicclassRunNotification{publicstaticvoidmain(String[]args)throwsIOException{// TODO(developer): Replace these variables before running the sample.finalStringprojectId="MY_PROJECT_ID";finalStringdatasetId="MY_DATASET_ID";finalStringpubsubTopicName="MY_TOPIC_NAME";finalStringquery="SELECT CURRENT_TIMESTAMP() as current_time, @run_time as intended_run_time, "+"@run_date as intended_run_date, 17 as some_integer";Map<String,Value>params=newHashMap<>();params.put("query",Value.newBuilder().setStringValue(query).build());params.put("destination_table_name_template",Value.newBuilder().setStringValue("my_destination_table_{run_date}").build());params.put("write_disposition",Value.newBuilder().setStringValue("WRITE_TRUNCATE").build());params.put("partitioning_field",Value.newBuilder().build());TransferConfigtransferConfig=TransferConfig.newBuilder().setDestinationDatasetId(datasetId).setDisplayName("Your Scheduled Query Name").setDataSourceId("scheduled_query").setParams(Struct.newBuilder().putAllFields(params).build()).setSchedule("every 24 hours").setNotificationPubsubTopic(pubsubTopicName).build();runNotification(projectId,transferConfig);}publicstaticvoidrunNotification(StringprojectId,TransferConfigtransferConfig)throwsIOException{try(DataTransferServiceClientdataTransferServiceClient=DataTransferServiceClient.create()){ProjectNameparent=ProjectName.of(projectId);CreateTransferConfigRequestrequest=CreateTransferConfigRequest.newBuilder().setParent(parent.toString()).setTransferConfig(transferConfig).build();TransferConfigconfig=dataTransferServiceClient.createTransferConfig(request);System.out.println("\nScheduled query with run notification created successfully :"+config.getName());}catch(ApiExceptionex){System.out.print("\nScheduled query with run notification was not created."+ex.toString());}}}

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

transfer_config_name="projects/1234/locations/us/transferConfigs/abcd"pubsub_topic="projects/PROJECT-ID/topics/TOPIC-ID"fromgoogle.cloudimportbigquery_datatransferfromgoogle.protobufimportfield_mask_pb2transfer_client=bigquery_datatransfer.DataTransferServiceClient()transfer_config=bigquery_datatransfer.TransferConfig(name=transfer_config_name)transfer_config.notification_pubsub_topic=pubsub_topicupdate_mask=field_mask_pb2.FieldMask(paths=["notification_pubsub_topic"])transfer_config=transfer_client.update_transfer_config({"transfer_config":transfer_config,"update_mask":update_mask})print(f"Updated config: '{transfer_config.name}'")print(f"Notification Pub/Sub topic: '{transfer_config.notification_pubsub_topic}'")

What's next

To search and filter code samples for other Google Cloud products, see theGoogle Cloud sample browser.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.