Load PayPal data into BigQuery

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Note: To get support or provide feedback for this feature, contact dts-preview-support@google.com.

You can load data from PayPal to BigQuery using thePayPal connector with the BigQuery Data Transfer Service. With thePayPal connector, you can schedule recurring transfer jobs thatadd your latest data from PayPal toBigQuery.

The PayPal connector supports production and sandbox PayPalaccounts.

Supported objects

PayPal object types	BigQuery-supported objects	Date filter support
Transactions	TransactionReports	Supported
	TransactionReportsCartInfoItemDetails	Supported
	TransactionReportsIncentiveDetails	Supported
Disputes	Disputes	Supported
	DisputeDetails	Supported
	DisputeTransactions	Supported
Payments	Payments	Supported
Payments	PaymentTransactions	Supported
Balance	Balance	Not supported
Products	Products	Not supported
Products	ProductDetails	Not supported
Invoices	Invoices	Supported

Limitations

PayPal data transfers are subject to the following limitations:

There can be a delay of several hours before PayPaltransactions become available through the PayPal API.
- We recommend scheduling subsequent data transfers at longer intervals (no more than one every hour)to prevent missing data.
The PayPal connector only supportstransactions data from thepast 3 years.
The PayPal connector only supportsdisputes data from the past6 months.
PayPal APIs use different page size limits for each data object. The PayPalconnector uses the maximum page size allowed byPayPal in a data transfer.
- However, some objects likePayments orPayment Transactions usesmaller page size limits. This can lead to slower data transfers, especiallywhen dealing with large datasets.

Before you begin

The following sections describe the steps that you need to take before youcreate a PayPal data transfer.

PayPal prerequisites

To enable data transfers from PayPal, you must have thefollowing:

You must have a PayPal Developer account. For moreinformation, seePayPal Developer Program.
Create a PayPal REST API app. For more information, seeGet started with PayPal REST APIs.
- In theApps & Credentials section, note the client ID and secret keyfor the app.
- In theFeatures section, enable theTransaction search andInvoicing API permissions.

Required BigQuery roles

To get the permissions that you need to create a transfer, ask your administrator to grant you theBigQuery Admin (roles/bigquery.admin) IAM role. For more information about granting roles, seeManage access to projects, folders, and organizations.

This predefined role contains the permissions required to create a transfer. To see the exact permissions that are required, expand theRequired permissions section:

Required permissions

The following permissions are required to create a transfer:

bigquery.transfers.update on the user
bigquery.datasets.get on the target dataset
bigquery.datasets.update on the target dataset

You might also be able to get these permissions withcustom roles or otherpredefined roles.

BigQuery prerequisites

Verify that you have completed all actions required toenable the BigQuery Data Transfer Service.
Create a BigQuery datasetto store your data.
If you intend to set up transfer run notifications for Pub/Sub,ensure that you have thepubsub.topics.setIamPolicy Identity and Access Management (IAM) permission. Pub/Subpermissions are not required if you only set up email notifications. For moreinformation, seeBigQuery Data Transfer Service run notifications.

Set up a PayPal data transfer

Add PayPal data into BigQuery by setting up atransfer configuration using one of the following options:

Console

Go to the Data transfers page in the Google Cloud console.
Go to Data transfers
ClickCreate transfer.
In theSource type section, forSource, selectPayPal.
In theData source details section, do the following:
- ForClient Id, enter the PayPal client ID. For more information, seePayPal prerequisites.
- ForClient Secret, enter the PayPal client secret key. For more information, seePayPal prerequisites.
- SelectIs Sandbox if you are using a sandbox PayPalaccount.
- ForStart Date, enter a date in the formatYYYY-MM-DD. The datatransfer loads PayPal data starting from this date.
  - If this field is left blank, this transfer defaults to retrieving datafrom the past 3 years.
  - For information about what objects support the start date filter, seeSupported objects.
- ForPayPal objects to transfer, enter the names of thePayPal objects to transfer, or clickBrowseand select the objects that you want to transfer.
In theDestination settings section, forDataset, select the datasetthat you created to store your data.
In theTransfer config name section, forDisplay name, enter a namefor the data transfer.
In theSchedule options section, do the following:
- In theRepeat frequency list, select an option to specify how oftenthis data transfer runs. To specify a custom repeat frequency, selectCustom. If you selectOn-demand, then this transfer runs when youmanually trigger the transfer.
- If applicable, select eitherStart now orStart at set time, andprovide a start date and run time.
Optional: In theNotification options section, do the following:
- To enable email notifications, toggleEmail notification to the onposition. When you enable this option, the transfer administratorreceives an email notification when a transfer run fails.
- To enablePub/Sub transfer runnotifications for thistransfer, togglePub/Sub notifications to the on position. You canselect yourtopic name, or you can clickCreate a topic to create one.
ClickSave.

bq

Enter thebq mk commandand supply the transfer creation flag--transfer_config:

bqmk--transfer_config--project_id=PROJECT_ID--data_source=DATA_SOURCE--display_name=DISPLAY_NAME--target_dataset=DATASET--params='PARAMETERS'

Where:

PROJECT_ID (optional): your Google Cloud project ID.If--project_id isn't supplied to specify a particular project, thedefault project is used.
DATA_SOURCE: the data source —paypal.
DISPLAY_NAME: the display name for the data transferconfiguration. The transfer name can be any value that lets youidentify the transfer if you need to modify it later.
DATASET: the target dataset for the transfer configuration.
PARAMETERS: the parameters for the created transferconfiguration in JSON format. For example:--params='{"param":"param_value"}'. The following are the parameters fora PayPal data transfer:
- assets: a list of PayPal objects to be included inthis transfer.
- connector.authentication.clientId: client ID of thePayPal application.
- connector.authentication.clientSecret: client secret of thePayPal application.
- connector.isSandbox: set value totrue if you are using a sandboxPayPal account, orfalse if you are using a productionPayPal account.
- connector.createdStartDate: (Optional) enter a date in the formatYYYY-MM-DD. The datatransfer loads PayPal data starting from this date.

For example, the following command creates a PayPal data transfer in thedefault project with all the required parameters:

bqmk\--transfer_config\--target_dataset=mydataset\--data_source=PayPal\--display_name='My Transfer'\--params='{"assets":  ["Payments", "TransactionReports"],          "connector.authentication.clientId": "112233445566",          "connector.authentication.clientSecret":"123456789",          "connector.isSandbox":"false",          "connector.createdStartDate":  "2025-01-01"}'

When you create a data transfer using the bq command-line tool, the transferconfiguration schedules data transfers once every 8 hours.

Note: You cannot configure notifications if you set up a transfer configuration using the bq command-line tool.

API

Use the projects.locations.transferConfigs.create method and supply an instance of theTransferConfig resource.

When you save the transfer configuration, the PayPalconnector automatically triggers a transfer run according to your scheduleoption. With every transfer run, the PayPal connectortransfers all available data from PayPal intoBigQuery.

To manually run a data transfer outside of your regular schedule, you can startabackfill run.

Data type mapping

The following table maps PayPal data types to the correspondingBigQuery data types.

PayPal data type	BigQuery data type
`String`	`STRING`
`Decimal`	`BIGNUMERIC`
`Boolean`	`BOOL`
`Datetime`	`TIMESTAMP`

Troubleshoot transfer setup

If you are having issues setting up your data transfer, seePayPal transfer issues.

Pricing

There is no cost to transfer PayPal data intoBigQuery while this feature is inPreview.

What's next

For an overview of the BigQuery Data Transfer Service, seeIntroduction to BigQuery Data Transfer Service.
To learn about managing transfer configurations, including how to obtaininformation, list configurations, and view run history, seeManage transfers.
Learn how toload data with cross-cloud operations.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-19 UTC.

Movatterモバイル変換