Load Salesforce Marketing Cloud data into BigQuery
You can load data from Salesforce Marketing Cloud to BigQuery using theBigQuery Data Transfer Service for Salesforce Marketing Cloud connector. With theBigQuery Data Transfer Service, you can schedule recurring transfer jobs thatadd your latest data from Salesforce Marketing Cloud toBigQuery.
Limitations
Salesforce Marketing Cloud data transfers are subject to the following limitations:
- A single transfer configuration can only support one data transfer run at a giventime. In the case where a second data transfer is scheduled to run before thefirst transfer is completed, then only the first data transfer completes whileany other data transfers that overlap with the first transfer is skipped.
- To avoid skipped transfers within a single transfer configuration, werecommend that you increase the duration of time between large datatransfers by configuring theRepeat frequency.
- To use a network attachment with this data transfer, you must firstcreate anetwork attachment by defining a static IPaddress.
- If your configured network attachment and virtual machine (VM) instance arelocated in different regions, there might be cross-region data movement whenyou transfer data from Salesforce Marketing Cloud.
Before you begin
The following sections describe the steps that you need to take before youcreate a Salesforce Marketing Cloud data transfer.
Salesforce Marketing Cloud prerequisites
You must have the following information when creating a Salesforce Marketing Clouddata transfer:
| Parameter Name | Description |
|---|---|
subdomain | The API subdomain, found in the base URI. For example, in the authentication base URIhttps://SUBDOMAIN.auth.marketingcloudapis.com/,SUBDOMAIN is your subdomain value. |
instance | The API server instance, found in the URL after you sign into the Salesforce Marketing Cloud application. The instance value includes `s` followed by a numeric value. For example, in the URL `https://mc.s4.exacttarget.com/`, the instance value is `s4`. For more information, seeFind the stack location for a Marketing Cloud account |
clientId | The client ID from the API integration. Navigate toSetup>Apps>Installed Packages, and then click the package name. The client ID is listed underComponents. |
clientSecret | The app integration client secret. Navigate toSetup>Apps>Installed Packages, and then click the package name. The client secret is listed underComponents. |
Salesforce Marketing Cloud objects to transfer | Compile a list of Salesforce Marketing Cloud objects to include in this transfer. You can select objects when youset up a transfer configuration. For a list of supported objects, seeSupported tables. |
Setup IP allowlist for Salesforce Marketing Cloud transfers
You must configure your Google Cloud environment and your Salesforce Marketing Cloudaccount to add specific IP addresses to the allowlist for data transfers.This ensures that Salesforce Marketing Cloud only accepts connections from a trusted,static IP address.
To do so, you must first set up and configure your Google Cloud network to usea static IP address:
- Set up a public network address translation(NAT) with a static IPaddress within your Virtual Private Cloud network. The CloudNAT must be configuredwithin the same region as the destination dataset for this data transfer.
- Set up a network attachmentwithin the same VPC network. This resource is used by theBigQuery Data Transfer Service to access private services.
Next, you mustadd the static IP address to the allowlist inSalesforce Marketing Cloud.When adding the range of the IP address, use the static IP address from yourGoogle Cloud public NAT for both the beginning and ending IP addresses for theIP range.
Once you have set up the IP ranges, you can now specify the static IP when youset up your transfer configuration by selecting your network attachment in theNetwork attachment field.
Install and configure Salesforce Marketing Cloud API integration package
You must install a server-to-server API integration package inSalesforce Marketing Cloud. You can do so in Salesforce Marketing Cloud by installing a newinstalled package, and specifying the componentAPI Integration>Server-to-Server. For more information, seeCreate and Install Packages.
Once you've installed the API integration package, you must add the followingpermissions scopes:
- Access:
Offline Access - Email:
Read - OTT:
Read - Push:
Read - SMS:
Read - Web:
Read - Documents and images:
Read - Saved Content:
Read - Journeys:
Read - Audiences:
Read - List and Subscribers:
Read - Date Extensions:
Read - File Locations
Read - Tracking Events:
Read - Callbacks:
Read - Subscriptions:
Read - Campaign:
Read - Assets:
Read - Accounts:
Read - OTT Channels:
Read - Users:
Read
For more information, seeAPI Integration Permission Scopes.
BigQuery prerequisites
- Verify that you have completed all actions required toenable the BigQuery Data Transfer Service.
- Create a BigQuery datasetto store your data.
- If you intend to set up transfer run notifications for Pub/Sub,ensure that you have the
pubsub.topics.setIamPolicyIdentity and Access Management (IAM) permission. Pub/Subpermissions are not required if you only set up email notifications. For moreinformation, seeBigQuery Data Transfer Service run notifications.
Required BigQuery roles
To get the permissions that you need to create a BigQuery Data Transfer Service data transfer, ask your administrator to grant you theBigQuery Admin (roles/bigquery.admin) IAM role on your project. For more information about granting roles, seeManage access to projects, folders, and organizations.
This predefined role contains the permissions required to create a BigQuery Data Transfer Service data transfer. To see the exact permissions that are required, expand theRequired permissions section:
Required permissions
The following permissions are required to create a BigQuery Data Transfer Service data transfer:
- BigQuery Data Transfer Service permissions:
bigquery.transfers.updatebigquery.transfers.get
- BigQuery permissions:
bigquery.datasets.getbigquery.datasets.getIamPolicybigquery.datasets.updatebigquery.datasets.setIamPolicybigquery.jobs.create
You might also be able to get these permissions withcustom roles or otherpredefined roles.
For more information, seeGrantbigquery.admin access.
Set up a Salesforce Marketing Cloud data transfer
Add Salesforce Marketing Cloud data into BigQuery by setting up atransfer configuration using one of the following options:
Console
Go to the Data transfers page in the Google Cloud console.
ClickCreate transfer.
In theSource type section, forSource, selectSalesforce Marketing Cloud.
In theData source details section, do the following:
- ForNetwork attachment, select a network attachment from the menu.Before you can use a network attachment with this data transfer, youmustcreate a network attachment by defining a static IP address.
- ForAPI Subdomain, enter thesubdomain of your authentication base URI.
- ForAPI instance, enter theAPI instance from the URLafter you sign in to the Marketing Cloud application.
- ForClient ID, enter theclient ID from your API integrationpackage.
- ForClient Secret, enter theclient secret from your API integrationpackage.
In theDestination settings section, forDataset, select thedataset that you created to store your data.
In theTransfer config name section, forDisplay name, enter aname for the data transfer.
In theSchedule options section, do the following:
- In theRepeat frequency list, select an option to specify howoften this data transfer runs. To specify a custom repeat frequency,selectCustom. If you selectOn-demand, then this transferruns when youmanually trigger the transfer.
- If applicable, select eitherStart now orStart at set time,and provide a start date and run time.
Optional: In theNotification options section, do the following:
- To enable email notifications, click theEmail notification toggle.When you enable this option, the transfer administrator receives anemail notification when a transfer run fails.
- To enablePub/Sub transfer run notificationsfor this transfer, click thePub/Sub notifications toggle. You canselect yourtopic name, or you can clickCreate a topic to create one.
ClickSave.
bq
Enter thebq mk commandand supply the transfer creation flag —--transfer_config.
bqmk--transfer_config--project_id=PROJECT_ID--data_source=DATA_SOURCE--display_name=DISPLAY_NAME--target_dataset=DATASET--params='PARAMETERS'
Replace the following:
- PROJECT_ID (optional): your Google Cloud project ID.If
--project_idisn't supplied to specify a particular project, thedefault project is used. - DATA_SOURCE: the data source (for example,
saphana). - DISPLAY_NAME: the display name for the transferconfiguration. The data transfer name can be any value that lets youidentify the transfer if you need to modify it later.
- DATASET: the target dataset for the transfer configuration.
- PARAMETERS: the parameters for the created transferconfiguration in JSON format. For example:
--params='{"param":"param_value"}'. The following are the parameters fora Salesforce Marketing Cloud transfer:connector.subdomain: The API subdomain.connector.instance: The API instance value.connector.authentication.oauth.clientId: The app ID name for the OAuth client.connector.authentication.oauth.clientSecret: The app secret for the OAuth client.assets: a list of the names of the Salesforce Marketing Cloudtables to be transferred from Salesforce Marketing Cloud as part ofthe transfer.
For example, the following command creates a Salesforce Marketing Cloud data transfer in thedefault project with all the required parameters:
bqmk--transfer_config--target_dataset=mydataset--data_source=salesforce_marketing--display_name='My Transfer'--params='{"connector.subdomain": "abcd", "connector.instance": "x", "connector.authentication.oauth.clientId": "1234567890", "connector.authentication.oauth.clientSecret":"ABC12345"}'
API
Use theprojects.locations.transferConfigs.create methodand supply an instance of theTransferConfig resource.
When you save the transfer configuration, the Salesforce Marketing Cloudconnector automatically triggers a transfer run according to your scheduleoption.
Supported tables
With every transfer run, the Salesforce Marketing Cloud connectortransfers all available data from Salesforce Marketing Cloud intoBigQuery into the following tables based on the REST interface:
AssetsCampaignAssetsCampaignsCategoriesEventDefinitionsFacebookMessengerPropertiesJourneyActivitiesJourneysLineMessengerPropertiesSendDefinitionsSubscriptionsDataExtensionDataExtensionObject_DATA_EXTENSION_NAMEEmailLinkSendListListSubscriberSubscriberTriggeredSendDefinition
To manually run a data transfer outside of your regular schedule, you can startabackfill run.
Data type mapping
The following table maps Salesforce Marketing Cloud data types to the correspondingBigQuery data types.
| Salesforce Marketing Cloud data type | BigQuery data type |
|---|---|
Boolean | BOOLEAN |
Number | INTEGER |
Text | STRING |
Decimal | FLOAT |
EmailAddress | STRING |
Phone | STRING |
Date | DATE |
DateTime | TIMESTAMP |
Locale | STRING |
Troubleshoot transfer setup
If you are having issues setting up a Salesforce Marketing Cloud data transfer, try thefollowing troubleshooting steps:
- Ensure that theconfigured authentication for the API integration packageis configured toServer-to-server.
- Ensure that the authentication app is configured with therequired permissionsunderScope.
Error messages
- Error:
invalid_grant. The client's IP address is unauthorized for this account. Allowlist the client's IP address in Marketing Cloud Administration. Resolution: Try one of the following steps:
- Enableall available IP addresses for Google Cloud resources.
- Disable the IP allow list. You can do this by logging in to theSalesforce Marketing Cloud application, then navigating to theSetup>Security Settingspage. ClickEdit, then configure theRestrict Logins by IP Address (IP Allowlisting)setting toIP Allowlisting Disabled.
- Error:
INVALID_ARGUMENT. TabletableName does not exist in asset TableName Resolution: Ensure that you have the correct scope permissions configuredin the Salesforce Marketing Cloud application. For more information, seeSalesforce Marketing Cloud prerequisites.
- Error:
FAILED_PRECONDITION: There was an issue connecting to API. Resolution: This error can occur when you include a network attachmentwith your transfer but have not configured your public NAT and set up your IPallow list. To resolve this error, follow the steps inCreate a network attachmentand follow the steps to create your network attachment by defining a static IPaddress.
Pricing
For pricing information about Salesforce Marketing Cloud transfers, seeData Transfer Service pricing.
What's next
- For an overview of the BigQuery Data Transfer Service, seeIntroduction to BigQuery Data Transfer Service.
- For information on using data transfers, including getting information about atransfer configuration, listing transfer configurations, and viewing atransfer's run history, seeManage transfers.
- Learn how toload data with cross-cloud operations.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-15 UTC.