Movatterモバイル変換


[0]ホーム

URL:


DOCS >REPLICATION >LOADING >DESTINATION GUIDES >MICROSOFT SQL SERVER DATA LOADING BEHAVIOR

Get started with a free trial today

Already have an account? Sign in

Microsoft SQL Server Data Loading Behavior

Learn how Stitch will load data from your integrations into Stitch’s Microsoft SQL Server destination.

In this guide, we’ll cover data loading scenarios involving:


Primary Key scenarios

Scenarios involving Primary Key columns.

IF

A table without a Primary Key is replicated.

THEN
  • Initial job: Table is created without Primary Key andNOT NULL constraints.
  • Subsequent jobs: If using Key-based Incremental or Log-based Incremental Replication, data will be added to the table in an Append-Only fashion.

    If using Full Table Replication, the table will be overwritten in its entirety during each job.

IF

A table with a single Primary Key is replicated.

THEN
  • Initial job: Table is created with Primary Key constraint. The Primary Key column hasNOT NULL constraint.
  • Subsequent jobs: If using Key-based Incremental or Log-based Incremental Replication, data will be de-duped based on the Primary Key column and upserted into the table.

    If using Full Table Replication, the table will be overwritten in its entirety during each job.

IF

A table with multiple Primary Keys is replicated.

THEN
  • Initial job: Table is created with Primary Key constraint. The Primary Key columns haveNOT NULL constraint.
  • Subsequent jobs: If using Key-based Incremental or Log-based Incremental Replication, data will be de-duped based on the Primary Key columns and upserted into the table.

    If using Full Table Replication, the table will be overwritten in its entirety during each job.

IF

The table’s Primary Key(s) is/are changed.

THEN

An error occurs and Stitch stops processing data for the table.

AND

The following error will display in theNotifications tab in Stitch:

FIX IT
  1. Drop the table in Microsoft SQL Server
  2. Reset the table in Stitch. This will queue a full re-replication of the table’s data, and re-create the table in Microsoft SQL Server with the correct data type for the Primary Key column.
IF

The table’s Primary Key(s) type is/are changed.

THEN

An error occurs and Stitch stops processing data for the table.

AND

The following error will display in theNotifications tab in Stitch:

FIX IT
  1. Drop the table in Microsoft SQL Server
  2. Reset the table in Stitch. This will queue a full re-replication of the table’s data, and re-create the table in Microsoft SQL Server with the correct data type for the Primary Key column.
IF

The number of Primary Keys in the table is changed.

THEN

An error occurs and Stitch stops processing data for the table.

FIX IT
  1. Drop the table in Microsoft SQL Server
  2. Reset the table in Stitch. This will queue a full re-replication of the table’s data, and re-create the table in Microsoft SQL Server with the correct data type for the Primary Key column.

Back to top


Replication Key scenarios

Scenarios involving Replication Keys and how data is loaded as a result.

IF

A table using Key-based Incremental Replication is replicated where the Replication Key column containsNULL values.

THEN
  • During the initial job, the table will be created and all rows will be replicated.
  • During subsequent jobs, only rows with populated Replication Keys will be replicated and persisted to Microsoft SQL Server.

Back to top


Object naming scenarios

Scenarios involving object identifiers in the destination, including naming limitations and transformations.

IF

A table name contains more characters than allowed by Microsoft SQL Server.

THEN

Microsoft SQL Server will reject all data for the table.

AND

The following error will display in theNotifications tab in Stitch:

Table name[TABLE] is too longforMicrosoft SQL Server

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

If possible, change the table name in the source to be less than Microsoft SQL Server’s character limit of 113 characters.

Use the_sdc_rejected table to identify the root of the issue.

IF

A column name contains more characters than allowed by Microsoft SQL Server.

THEN

Microsoft SQL Server will reject columns with names that exceed the column character limit. Other columns in the table will persist to Microsoft SQL Server.

AND

The following error will display in theNotifications tab in Stitch:

Column name[COLUMN] is too longforMicrosoft SQL Server

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

If possible, change the column name in the source to be less than Microsoft SQL Server’s character limit of 128 characters.

Use the_sdc_rejected table to identify the root of the issue.

IF

Two columns are replicated that canonicalize to the same name.

THEN

For example: A table containing bothCustomerId andcustomerid columns.

Microsoft SQL Server will reject the records and create a log for the rejected records in the_sdc_rejected table in that integration’s schema.

AND

The following error will display in theNotifications tab in Stitch:

Field collision on[COLUMN_NAME]

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

If possible, re-name one of the columns in the source so that both column names will be unique when replicated to Microsoft SQL Server.

Use the_sdc_rejected table to identify the root of the issue.

IF

A column is replicated that has a mixed-case name.

THEN

Microsoft SQL Server will convert letters to lowercase. For example:

Columns in SourceColumns in Microsoft SQL Server
CuStOmErIdcustomerid
customerIDcustomerid
IF

A column is replicated that has a name with spaces.

THEN

Microsoft SQL Server will convert spaces to underscores, with the exception of leading and trailing spaces. For example:

Columns in SourceColumns in Microsoft SQL Server
customer idcustomer_id
CUSTOMER IDcustomer_id
IF

A column name contains leading or trailing spaces.

THEN

Microsoft SQL Server will trim leading and trailing spaces. For example:

Columns in SourceColumns in Microsoft SQL Server
CUSTOMER IDcustomer_id
customeridcustomerid
IF

A column is replicated with a name that contains unsupported special characters.

THEN

Microsoft SQL Server will convert special characters to underscores, with the exception of the following characters:@,$ and#. For example:

Columns in SourceColumns in Microsoft SQL Server
customer!idcustomer_id
customer#idcustomer#id
IF

A column name starts with a special character.

THEN

Microsoft SQL Server will remove all leading special characters with the exception of leading underscores. For example:

Columns in SourceColumns in Microsoft SQL Server
!customeridcustomerid
_customerid_customerid
IF

A column name is equal to a reserved word in Microsoft SQL Server.

THEN

Microsoft SQL Server will quote column names which are reserved words. For example:

Columns in SourceColumns in Microsoft SQL Server
order"order"

Back to top


Table scenarios

Scenarios involving table creation and modification in the destination.

IF

A table contains entirelyNULL columns.

THEN

No table is created in Microsoft SQL Server. At least one column must have a non-NULL value for Stitch to create a table in Microsoft SQL Server.

IF

A table arrives with more columns than Microsoft SQL Server allows.

THEN

Microsoft SQL Server will reject all data for the table.

AND

The following error will display in theNotifications tab in Stitch:

ERROR: too many columns

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

If possible, deselect some columns to allow Stitch to load data into Microsoft SQL Server for the table. Microsoft SQL Server has a limit of 1,024 columns per table.

Use the_sdc_rejected table to identify the root of the issue.

Back to top


Data typing scenarios

Scenarios involving various data types, including how data is typed and structured in the destination.

IF

Stitch detects multiple data types for a single column.

THEN

To accommodate data of varying types, Stitch will create multiple columns to ensure data is loaded with the correct type. In the destination, this will look like the column has been “split”.

For example: Stitch first detected thatorder_confirmed containedBOOLEAN data, but during a subsequent job, detectedSTRING values. To accommodate data of varying types, Stitch will:

  1. Store data for the original data type in the original column. In this example, onlyBOOLEAN values will be stored inorder_confirmed. The name of the original column will not change.

  2. Create additional columns to store the other data types - one for each data type detected - and append the data type to the column name. In this example, aorder_confirmed__st column will be created to storeSTRING values.

IF

Data is replicated to Microsoft SQL Server that is nested, containing many top-level properties and potentially nested sub-properties.

THEN

Nested data will be maintained.

If the data contains nested arrays, Microsoft SQL Server will reject the data.

AND

The following error will display in theNotifications tab in Stitch:

JSON table encountered too many errors, giving up. Rows: 1; errors: 1. JSON parsing errorinrow starting at position 0:. Nested arrays not allowed.
FIX IT

If possible, remove the offending record from the source.

IF

AVARCHAR column is replicated to Microsoft SQL Server.

THEN

Microsoft SQL Server will store allVARCHAR data asNVARCHAR(450).

IF

VARCHAR data is loaded that exceeds the current maximum size for the column.

THEN

Microsoft SQL Server will store allVARCHAR data asNVARCHAR(MAX).

IF

A column containing date data with timezone info is replicated to Microsoft SQL Server.

THEN

Microsoft SQL Server will store the value asDATETIMEOFFSET(3).

IF

A column contains timestamp data that is outside Microsoft SQL Server’s supported range.

THEN

Microsoft SQL Server will reject the records that fall outside the supported range.

AND

The following error will display in theNotifications tab in Stitch:

timestamp out of rangeforMicrosoft SQL Server on[TIMESTAMP]

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

To resolve the error, offending values in the source must be changed to be within Microsoft SQL Server’s timestamp range.

Use the_sdc_rejected table to identify the root of the issue.

IF

A column contains integer data.

THEN

Integer values will be loaded to Microsoft SQL Server as the data typeINT.

IF

A column contains integer data that is outside Microsoft SQL Server’s supported range.

THEN

Microsoft SQL Server will reject the records that fall outside the supported range.

AND

The following error will display in theNotifications tab in Stitch:

integer out of rangeforMicrosoft SQL Server on[INTEGER]

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

To resolve the error, offending values in the source must be changed to be within Microsoft SQL Server’s limit for integers.

Use the_sdc_rejected table to identify the root of the issue.

IF

A column contains decimal data.

THEN

Decimal values will be loaded to Microsoft SQL Server as the data typeDECIMAL(38,6).

IF

A column contains decimal data that is outside Microsoft SQL Server’s supported range.

THEN

Microsoft SQL Server will reject the records that fall outside the supported maximum range for theDECIMAL(38,6) data type.

AND

The following error will display in theNotifications tab in Stitch:

Rejected records will be logged in the_sdc_rejected table of the integration's schema.Learn more.

FIX IT

To resolve the error, offending values in the source must be changed to be within Microsoft SQL Server’s limit for decimals.

Use the_sdc_rejected table to identify the root of the issue.

Back to top


Schema change scenarios

Scenarios involving schema changes in the source or structural changes in the destination.

IF

A new column is added in table already set to replicate.

THEN

If the column has at least one non-NULL value in the source, the column will be created and appended to the end of the table in Microsoft SQL Server.

Note: If the table using either Key- or Log-based Incremental Replication, backfilled values for the column will only be replicated if:

  1. The records’ Replication Key values are greater than or equal to the last saved maximum Replication Key value for the table, or
  2. The table is reset and a historical re-replication is queued.
IF

A new column is added by you to a Stitch-generated table in Microsoft SQL Server.

THEN

Columns may be added to tables created by Stitch as long as they are nullable, meaning columns don’t haveNOT NULL constraints.

IF

A column is deleted at the source.

THEN

How a deleted column is reflected in Microsoft SQL Server depends on the Replication Method used by the table:

  • Key-based Incremental: The column will remain in the destination, and defaultNULL values will be placed in it going forward.

  • Log-based Incremental: Changes to a source table - including adding or removing columns, changing data types, etc. - require manual intervention before replication can continue. Refer to theLog-based Incremental Replication documentation for more info.

  • Full Table: The column will remain in the destination, and defaultNULL values will be placed in it going forward.

IF

You remove a column from a Stitch-replicated table in your destination.

THEN

The result of deleting a column from a Stitch-generated table depends on thetype of column being removed:

  • Primary Key columns: Changing a table’s Primary Key(s) is not permitted in Microsoft SQL Server. If Primary Key columns are changed, Stitch will stop processing data for the table.

  • General columns: If new data is detected for the removed column, Stitch will re-create it in Microsoft SQL Server. This refers to all columns that are not prepended by_sdc or suffixed by a data type. For example:customer_zip, but notcustomer_zip__st.

    Note: An integration must support selecting columns AND you must deselect the column in Stitch for the column removal to be permanent.

  • _sdc columns: Removing a Stitch replication column will prevent Stitch from loading replicated data into Microsoft SQL Server.

  • Columns with data type suffixes: Removing a column created as result of accommodatingmultiple data types will prevent Stitch from loading replicated data into the table. This applies to columns with names such as:customer_zip__st,customer_zip__int, etc.

Back to top


Destination changes

Scenarios involving modifications made to the destination, such as the application of workload/performance management features or user privilege changes.

IF

Indices are applied to Stitch-generated columns in the destination.

THEN

Stitch will respect the index application.

IF

You switch to a different destination of the same type.

THEN

This means the destination type is still Microsoft SQL Server, Stitch may just be connected a different database in Microsoft SQL Server.

  • For tables using Key-based or Log-based Incremental Replication, replication will continue using the Replication’s Key last saved maximum value. To re-replicate historical data, resetting Replication Keys is required.
  • For tables using Full Table Replication, the table will be fully replicated into the new destination during the next successful job.
  • For webhook integrations, some data loss may be possible due to the continuous, real-time nature of webhooks. Historical data must either be backfilled or re-played.

Back to top


RelatedTroubleshooting

Questions? Feedback?

Did this article help? If you have questions or feedback, feel free tosubmit a pull request with your suggestions,open an issue on GitHub, orreach out to us.


[8]ページ先頭

©2009-2026 Movatter.jp