Already have an account? Sign in

Syncing New and Additional Columns on Already-Syncing Tables

When you add a new column to a table in your data source, what happens in Stitch? What about syncing additional columns on already-syncing tables? Depending on the type of integration and theReplication Method the table uses, there are a few possible outcomes.

New Columns in the Data Source & Stitch

When a new column is added in a data source, Stitch will first need to perform astructure sync to detect it.

Note that:

• It may take some time for Stitch to detect the new column and display it in the Integration Details page.
• The following applies to both database and non-webhook SaaS integrations.

Already-Syncing Tables

After the structure sync completes, the column willbe automatically synced and,if theintegration supports whitelisting columns, display in the Integration Details page. Data for the column will then replicate based on the table’s Replication Method.

Keep in mind that for Incremental tables, you may need tobackfill existing rows if the addition of new data hasn’t updated the table’sReplication Key.

Non-Syncing Tables

If the table isn’t currently selected to sync, the new column will display inside Stitch (if theintegration supports whitelisting columns)but will have to be manually set to sync.

Syncing Additional Columns on Already-Syncing Tables

In this case, we aren’t talking about brand new columns in the data source,but previously existing columns that have never been set to sync in Stitch. How Stitch handles the syncing of additional rows depends on the table’s Replication Method.

Full Table Replication

For tables using Full Table Replication, data in the newly-synced column will be available forall rows - including new and existing - the next time the table is successfully replicated.

Incremental Replication

For tables using Incremental Replication, data in the newly-synced column will be availableonly for rows added AFTER the column is synced. Existing rows must be backfilled to make the data available.

Getting newly-synced column data into existing rows requires a full re-replication of the table. Because this can significantly impact your row count and we don’t want to re-replicate data without your say-so, we leave inserting newly-synced column data into existing rows up to you.

Backfilling existing rows

If you need to backfill already-replicated rows with data from the newly selected column, you canreset the table’s Replication Key. This will force a full re-replication of the table, populate the column in existing rows, and replicate new records.

Resetting database integration Replication Keys

Replication Keys in database integrations can be reset at theintegration or thetable level.

• At theintegration level, the reset will clear the replication key value for ALL tables AND queue a full re-replicationfor all tables in the integration.
• At thetable level, the reset will clear the replication key value AND queue a full re-replicationfor that table only.

To reset Replication Keys, do the following:

1. Click into the integration from the Stitch Dashboard page.
2. Next:
   • To reset the entire integration: Click theSettings link andskip to step 3.
   • To reset a table: Locate the table you want and click into it. Click theTable Settings link, located near the top right corner, andproceed to step 3.
3. Scroll down to theReset Replication Keys section.
4. Click theReset Keys button.
5. When prompted, clickOK to confirm.
6. ASuccess! message will display at the top of the page.

At this point, a full re-replication of the integration or table will be queued.Note: If there is a large volume of data to be replicated, it may take some time before you see the changes in your data warehouse.

Resetting SaaS integration Replication Keys

Resetting the Replication Keys for a SaaS integration is done by changing theHistorical Sync date in the Integration Settings page. When this date is changed, all saved values will be overwritten AND a full re-replication of the integration will be queued.

Note: This feature may not be available for some integrations. Because this approach uses date-based replication, some integrations may be incompatible. For example: Pardot doesn’t support date-based replication, meaning this feature will not be available for Pardot connections.

Changing the Historical Sync date has its own set of considerations and gotchas. Please refer to theSyncing Historical SaaS Data guide for more info.

Did this article help? If you have questions or feedback, feel free tosubmit a pull request with your suggestions,open an issue on GitHub, orreach out to us.