Storage Transfer Service uses metadata available from the source storage system,such as checksums and file sizes, to ensure that data written toCloud Storage is the same data read from the source.

When checksum metadata is available

If checksum metadata is available on the source, Storage Transfer Service usesend-to-end checksum validation. If the checksum metadata on the source storagesystem indicates that the data Storage Transfer Service received doesn't match thesource data, Storage Transfer Service records a failure for the transfer operation.Examples of storage systems that include checksum metadata include most Amazon Simple Storage Service (Amazon S3)and Microsoft Azure Blob Storage objects (with some exceptions)and HTTP transfers (checksum metadata are provided by the user).

When checksum metadata is unavailable

When agents can run near the source

If checksum metadata isn't available from the underlying source storage systembutagents can be run locallynear the source storage system, Storage Transfer Service attempts to read thesource data and compute a checksum before sending the data toCloud Storage. This occurs when moving data from file systems toCloud Storage.

When agents can't run near the source

If checksum metadata isn't available from the underlying source storage system,and agents can't be run locally near the source storage system,Storage Transfer Service uses available metadata, such as file size, to validate thatthe file copied to Cloud Storage matches the source file.

Storage Transfer Service also computes a checksum of the data "on-the-fly" as it'sbeing transferred. Once the data is stored in Cloud Storage,Storage Transfer Service compares the checksum it computed against the final checksumreported by Cloud Storage. If the checksums don't match, the taskfails with aDATA_LOSS error.

On-the-fly checksumming is used for transfers from sources that may not providechecksums, such as:

• Amazon S3 multi-part objects

• Amazon S3 objects stored with server-side encryption with keys stored in AWS Key Management Service (SSE-KMS)

• Microsoft Azure Blob Storage Put Block List

After transfer checks

After your transfer is complete, we recommend performing additional dataintegrity checks to validate that:

• The correct version of the files are copied, for files that change atthe source.
• The correct set and number of files are copied, to verify that you'veset up the transfer jobs correctly.
• The files were copied correctly, by verifying the metadata on the files,such as file checksums, file size, and so forth.