Load Display & Video 360 data into BigQuery

You can load data from Display & Video 360 to BigQuery using theBigQuery Data Transfer Service for Display & Video 360 connector. With theBigQuery Data Transfer Service, you can schedule recurring transfer jobs thatadd your latest data from your Display & Video 360 toBigQuery.

Connector overview

The BigQuery Data Transfer Service for the Display & Video 360 connector supports the following options for your data transfer.

Supported configuration data

In addition to the reporting data, BigQuery Data Transfer Service also transfersthe following configuration data from Display & Video 360. Configuration datais retrieved fromDisplay & Video 360 API v3.

• Partner
• Advertiser
• LineItem
• LineItemTargeting
• Campaign
• CampaignTargeting
• InsertionOrder
• InsertionOrderTargeting
• AdGroup
• AdGroupTargeting
• AdGroupAd
• Creative

For more information about each type of configuration data, see the followinglinks:

• About Partners
• Create an advertiser
• Create a line item
• Create a campaign
• Create an insertion order
• About YouTube & partners line items
• Manage creatives

Data ingestion from Display & Video 360 transfers

Refresh windows

Arefresh window is the number of days that a data transfer retrieves datawhen a data transfer occurs. For example, if the refresh window is three daysand a daily transfer occurs, the BigQuery Data Transfer Service retrieves all data fromyour source table from the past three days. In thisexample, when a daily transfer occurs, the BigQuery Data Transfer Service creates a newBigQuery destination table partition with a copy of your source table datafrom the current day, then automatically triggers backfill runs to update theBigQuery destination table partitions with your source table data from thepast two days. The automatically triggered backfill runs will either overwriteor incrementally update your BigQuery destination table,depending on whether or not incremental updates are supported in theBigQuery Data Transfer Service connector.

When you run a data transfer for the first time, the data transfer retrieves allsource data available within the refresh window. For example, if the refreshwindow is three days and you run the data transfer for the first time, theBigQuery Data Transfer Service retrieves all source data within three days.

To retrieve data outside the refresh window, such as historical data, or torecover data from any transfer outages or gaps, you can initiate or schedule abackfill run.

Before you begin

Review the following prerequisites and information before you create a Display & Video 360data transfer.

Prerequisites

• Verify that you have completed all actions required toenable the BigQuery Data Transfer Service.
• Create a BigQuery dataset to storeDisplay & Video 360 data.
• Ensure that you have either your Display & Video 360Partner IDorAdvertiser ID.The partner ID is the parent in the hierarchy.
• Ensure that you haveread permissionsto access partner or advertiser data from the Display and Video API.

• Ensure that your organization has access to Display & Video 360 DataTransfer v2 (Display & Video 360 DTv2) files. These files are delivered bythe Display & Video 360 team in a Cloud Storage bucket. Requesting accessto the Display & Video 360 DTv2 files depends on whether or not you have adirect contract with Display & Video 360. In both cases, additional chargesmight apply.

  • If you have a contract with Display & Video 360,contact Display & Video 360 supportto set up Display & Video 360 DTv2 files.
  • If you don't have a contract with Display & Video 360, contact your agencyfor access to Display & Video 360 DTv2 files.
  • After completing this step, you will receive either of the followingCloud Storage bucket name, depending if your setup is for a partneror an advertiser:
    • gs://dcdt_-dbm_partnerPARTNER_ID
    • gs://dcdt_-dbm_advertiserADVERTISER_ID
  Note: The Google Cloud team does not have the ability to generate orgrant access to Display & Video 360 DTv2 files on your behalf. ContactDisplay & Video 360supportor your agency for access to Display & Video 360 DTv2 files.

• To set up transfer run notifications for Pub/Sub, youmust havepubsub.topics.setIamPolicy permissions. For more information,seeBigQuery Data Transfer Service run notifications.

Find your Display & Video 360 ID

To retrieve your Display & Video 360 ID, navigate to the Cloud StorageBuckets page in the Google Cloud console and examine the files in yourDisplay & Video 360 data transfer Cloud Storage bucket. The Display & Video 360ID is used to match files in the providedCloud Storage bucket. The ID is embedded in the filename, not theCloud Storage bucket name. For example:

• In a file nameddbm_partner123_activity_*, the ID is123.
• In a file nameddbm_advertiser567_activity_*, the ID is567.

Finding your filename prefix

In some cases, the files in your Cloud Storage bucket might have custom,nonstandard file names that were set up for you by the Google Marketing Platformservices team. For example:

In a file nameddbm_partner123456custom_activity_*, the prefix isdbm_partner123456custom.

For any assistance regarding filename prefixes, contactDisplay & Video 360 support.

Set up a Display & Video 360 data transfer

Select one of the following options:

bqmk--transfer_config\--project_id=PROJECT_ID\--target_dataset=DATASET\--display_name=NAME\--params='PARAMETERS'\--data_source=DATA_SOURCE

bqmk--transfer_config\--target_dataset=mydataset\--display_name='My Transfer'\--params='{"bucket":"dcdt_-dbm_partner123456","displayvideo_id": "123456","file_name_prefix":"YYY"}'\--data_source=displayvideo

Query your data

When your data is transferred to BigQuery, the data is written toingestion-time partitioned tables. For more information, seeIntroduction to partitioned tables.

We recommend that you query the auto-generated views instead of querying thetables directly. However, if you want to query your tables directly, you mustuse the_PARTITIONTIME pseudocolumn in your query. For moreinformation, seeQuerying partitioned tables.