Recommended use

You should use Amazon S3 for uploads of data to ODP on a scheduled basis from external systems. Examples of recurring uploads include:

• Product feeds
• Customer updates
• List subscriptions
• Consent updates

🚧

Important

CSV imports using Amazon S3 must adhere to the file format and name requirements outlined inImport data using CSV.

Generate Amazon S3 credentials

To use Amazon S3, you need the bucket (location) you want to access and the access key/secret for authentication. To retrieve these:

1. Go toData Setup > Integrations.

2. Select theAWS integration.

3. ClickGenerate Access Keys.

   

Locate your access keys and bucket URLs

YourKey ID andSecret Access Key display belowAccess Keys inData Setup > Integrations > AWS, which you can copy and paste into thecommand-line interface or athird-party application to complete the Amazon S3 integration with ODP.

Your Amazon S3 bucket URLs also display here. All clients have an Amazon S3 bucket for imports and another for exports. The structure of the bucket URLs is shown below:

• Data imports –s3://zaius-incoming/your_tracker_id
• Data exports –s3://zaius-outgoing/your_tracker_id

For additional help from AWS, you can install theAWS CLI tools and use theS3 command line reference guide.

🚧

Important

Data in the Amazon S3 buckets expires after 7 days.

Import from Amazon S3 to ODP using command-line

After you place a file in your Zaius-incoming bucket, ODP automatically imports it. You can fully automate the import process if you regularly create data for imports and can schedule to add that data to your S3 bucket.

Use the following command to copy a local file to Amazon S3:

aws s3 cp zaius\_customers.csv s3://zaius-incoming/<your tracker ID>/ --sse

Use the following command to copy a directory of files to Amazon S3:

aws s3 sync /tmp/yourlocaldir/ s3://zaius-incoming-temp/<your tracker ID>/ --sse

Export from ODP to Amazon S3 using command-line

1. Run aStart Export Job API request for your desired export format (CSV or parquet), delimiter (comma, tab, pipe), and objects.
2. Copy thepath value from the API response body. For example:

   s3://zaius-outgoing/lz3CnPijk15xYhTw7DU4wx/data-exports/3a44cik3-e981-53bf-6499-f9fc6851fae

3. Run the following command, replacing<PATH> with thepath value you copied in step 2. This retrieves all Amazon S3 files for the export you requested in step 1.

   aws s3 cp <PATH> . --recursive --sse

   📘

   Note

   Using the example from step 2:

   aws s3 cp s3://zaius-outgoing/lz3CnPijk15xYhTw7DU4wx/data-exports/3a44cik3-e981-53bf-6499-f9fc6851fae . --recursive --sse

This command outputs the contents of your requested export ID to your current directory. To specify a location, replace. with the directory path. For example, to output the directory to your desktop, specify one of the following, depending on your operating system:

• OS X –/Users/<name>/Desktop
• Windows –C:\\Users\\<name>\\Desktop (Windows)

📘

Note

ODP exports are a set of files in a directory folder identified by the export ID you provide. Be sure you transfer the entire directory to get the entirety of the export.

Import/export from Amazon S3 using a third-party application

You can use whichever third-party application you prefer. Common developer tools you can use to perform uploads to Amazon S3 include:

• Cyberduck (Windows and Mac) – marketer friendly
• AWS CLI (Windows, Mac, and Linux)
• AWS SDKs (Java, Python, Node.js, PHP, and more)

The following instructions are for Cyberduck, which is a free cloud storage browser that you can use with Amazon S3.

1. Download and launch a free tool, likeCyberduck.

2. In Cyberduck, expandAction and selectNew Bookmark.

3. SelectAmazon S3 from the drop-down list at the top of the pop-up window.

4. (Optional) Enter aNickname.

5. Enter theAccess Key ID (theKey ID from ODP).

6. Enter theSecret Access Key (theSecret Access Key from ODP).

7. ExpandMore Options and enter thePath (theData Exports bucket URL from ODP). Removes3:/ from the beginning of the URL, leaving only one forward slash.

   📘

   Note

   To locate your exact URL, go toAccount Settings > Integrations > AWS. UnderBucket URL(s), copy theData Exports value.

8. Close the pop-up window to save the new bookmark.

📘

Note

For information about using Cyberduck to import AWS files, seetheir documentation.

Updated 25 days ago