Load Google Merchant Center data into BigQuery

Preview

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Note: To get support or provide feedback for Google Merchant Center transfers with BigQuery Data Transfer Service, contactgmc-transfer-preview@google.com.

You can load data from Google Merchant Center to BigQuery using theBigQuery Data Transfer Service for Google Merchant Center connector. With theBigQuery Data Transfer Service, you can schedule recurring transfer jobs thatadd your latest data from Google Merchant Center toBigQuery.

Supported reports

The BigQuery Data Transfer Service for Google Merchant Center supports the following data:

Products and product issues

The products and product issues report includes product data provided to the Google Merchant Center through feeds or using the Content API for Shopping. This report also includes item level issues detected by Google for your products. You can view product and product issues data in theGoogle Merchant Center or by querying theContent API for Shopping. For information on how this data is loaded into BigQuery, see the Google Merchant Centerproduct table schema.

Regional Inventories

The regional inventories report includes additional product data aboutregional availability and pricing overrides of your products.For informationon how this data is loaded into BigQuery, see the Google Merchant Centerregional inventories tableschema.

Local Inventories

The local inventories report includes additional product data about localinventory of your products. This report contains data on local pricing,availability, quantity, pick-up and in-store product location. For informationon how this data is loaded into BigQuery, see the Google Merchant Centerlocal inventories tableschema.

Performance

The performance report provides granular segmentation of your performance dataacross both Ads and Free Listings. For information on how this data is loadedinto BigQuery, see the Google Merchant Centerperformance tableschema.

Best Sellers

The Best sellers report provides the same data found in the Google Merchant Center UI and lets you backfill the data across countries or categories forup to 2 years. This includes data about the most popular products and brands inShopping ads and unpaid listings, as well as whether or not you have them inyour inventory. This report is based on thebest sellersreport available through Google Merchant Center. For information on how this data isloaded into BigQuery, see the Google Merchant Centerbestsellers tables schemas.

Price Competitiveness

Formerly known as the price benchmarks report, the price competitiveness report includes product level attributes and price benchmark data and is basedon the same definitions as theprice competitivenessreport available through Google Merchant Center. For information on how this data is loaded into BigQuery, see the Google Merchant Centerprice competitiveness table schema.

Price Insights

Use the price insights report to see suggested sale prices for yourproducts, and predictions for the performance that you can expect if you updateyour products' prices. Using the price insights report can help you price yourproducts more effectively. For more information on how to use the data in thisreport, seeImprove product pricing with the price insightsreport for moreinformation on how to use the data in this report. For information on how thisdata is loaded into BigQuery, see the Google Merchant CenterPrice Insights tableschema.

Product Targeting

Enable the Product Targeting report when you set up a transfer to expose Adstargeting information when you load data from Google Shopping intoBigQuery. For information on how the data is loaded intoBigQuery, see the Google Merchant CenterProduct Targeting table schema.

Reporting options

The BigQuery Data Transfer Service for Google Merchant Center supports the followingreporting options:

Reporting option	Support
Schedule	Configurable to daily, weekly, monthly, or custom. By default, this is set to daily when the transfer is created. The minimum interval between transfers is 6 hours.

Data ingestion

When you transfer data from Google Merchant Center into BigQuery, thedata is loaded into BigQuery tables that are partitioned by date.The table partition that the data is loaded into corresponds to the date fromthe data source. If you schedule multiple transfers for the same date,BigQuery Data Transfer Service overwrites the partition for that specific date withthe latest data. Multiple transfers in the same day or running backfills don'tresult in duplicate data, and partitions for other dates are not affected.

Multi-client account (MCA) support

Existing customers with multiple Merchant IDs are encouraged to configure aparentMulti-Client Account (MCA).Configuring an MCA lets you create a single transfer for all your MerchantIDs.

Using Google Merchant Center MCAs provides several benefits over usingindividual Merchant IDs:

• You no longer need to manage multiple transfers to transfer reporting data formultiple Merchant IDs.
• Queries involving multiple Merchant IDs are much simpler to write because allMerchant ID data is stored in the same table.
• Using MCAs alleviates potential BigQuery load job quota issuesbecause all your Merchant ID data is loaded in the same job.

One possible disadvantage of using MCAs is that your subsequent query costs arelikely to be higher. Because all of your data is stored in the same table,queries that retrieve data for an individual Merchant ID must still scan theentire table.

Note: The BigQuery Data Transfer Service pulls reports for all listed Merchant IDs. Ifthere are no products in Google Shopping for a specific day, you may not seeMerchant IDs in the BigQuery table.

If you are using an MCA, the MCA ID is listed underaggregator_id and theindividual sub-accounts are listed undermerchant_id. For accounts thatdon't use an MCA,aggregator_id is set tonull.

Limitations

Some reports might have their own constraints, such as different windows ofsupport for historical backfills. The following sections describe thelimitations for each report.

Historical Backfills Support

Not all reports support historical backfills in the same way. The following area list of reports and the level of support for historical backfills.

• Products and Product Issues - 14 days
• Local Inventories - 14 days
• Regional Inventories - 14 days
• Performance - 2 years
• Best Sellers - 2 years
• Price Competitiveness - No backfill support
• Price Insights - No backfill support

Automatic Backfill Transfer Runs

The Performance report can have latencies in "today's" data. Therefore, when adata export is requested, data might update up to 3 days in the past to accountfor corrections.

To support this functionality, whenever a transfer is triggered on any report,two more transfer runs are created fortoday - 1 andtoday - 2. Thesetransfer runs only affect the Performance table; other tables are not impacted.

The automatic backfills can't be disabled.