Choosing a task mode for your data transfer
Your AWS DataSync task can run in one of the following modes:
Enhanced mode – Transfer virtually unlimited numbers of objects with higher performance than Basic mode. Enhanced mode tasks optimize the data transfer process by listing, preparing, transferring, and verifying data in parallel. Enhanced mode is currently available for transfers between Amazon S3 locations, transfers between Azure Blob and Amazon S3 without an agent, and transfers between other clouds and Amazon S3 without an agent.
Basic mode – Transfer files or objects between AWS storage and all other supported DataSync locations. Basic mode tasks are subject toquotas on the number of files, objects, and directories in a dataset. Basic mode sequentially prepares, transfers, and verifies data, making it slower than Enhanced mode for most workloads.
Understanding task mode differences
The following information can help you determine which task mode to use.
| Capability | Enhanced mode behavior | Basic mode behavior |
|---|---|---|
| Performance | DataSync lists, prepares, transfers, and verifies your data in parallel. Provides higher performance than Basic mode for most workloads (such as transferring large objects) | DataSync prepares, transfers, and verifies your data sequentially. Performance is slower than Enhanced mode for most workloads |
| Number of items in a dataset that DataSync can work with per task execution | Virtually unlimited numbers of objects | Quotas apply |
More counters and metrics than Basic mode, such as the number of objects that DataSync finds at your source location and how many objects are prepared during each task execution | Less counters and metrics than Enhanced mode | |
| Structured logs (JSON format) | Unstructured logs | |
| Currently for transfers between Amazon S3 locations, transfers between Azure Blob and Amazon S3 without an agent, and transfers between other clouds and Amazon S3 without an agent. | For transfers between all locations that DataSync supports | |
| DataSync verifies only transferred data | DataSync verifies all data by default | |
| Bandwidth limits | Not applicable | Supported |
| Cost | For more information, see theDataSync pricing page | For more information, see theDataSync pricing page |
| Failure handling for unsupported object tags | For cloud storage transfers to or from locations that don't support object tagging, task execution will fail immediately if theObjectTags option is unspeficied or set toPRESERVE. | For cloud storage transfers to or from locations that don't support object tagging, task execution will run normally, but will report per-object failures for tagged objects if theObjectTags option is unspecified or set toPRESERVE. |
Choosing a task mode
You can choose Enhanced mode only for transfers between Amazon S3 locations, transfers between Azure Blob and Amazon S3 without an agent, and transfers between other clouds and Amazon S3 without an agent. Otherwise, you must use Basic mode. For example, a transfer from an on-premisesNFS location to an S3 location requires Basic mode.
Your task options and performance might vary depending on the task mode you choose. Once you create your task, you can't change the task mode.
- Required permissions
To create an Enhanced mode task, the IAM role that you're using DataSync with must have the
iam:CreateServiceLinkedRolepermission.For your DataSync user permissions, consider usingAWSDataSyncFullAccess. This is an AWS managed policy that provides a user full access to DataSync and minimal access to its dependencies.
Open the AWS DataSync console athttps://console.aws.amazon.com/datasync/.
In the left navigation pane, expandData transfer, then chooseTasks, and then chooseCreate task.
Configure your task's source and destination locations.
For more information, seeWhere can I transfer my data with AWS DataSync?
ForTask mode, choose one of the following options:
Enhanced
Basic
For more information, seeUnderstanding task mode differences.
While still on theConfigure settings page, choose other task options or use the default settings.
You might be interested in some of the following options:
Configure how tohandle file metadata andverify data integrity.
Monitor your transfer withtask reports orAmazon CloudWatch Logs.
When you're done, chooseNext.
Review your task configuration, then chooseCreate task.
In your AWS CLI settings, make sure that you're using one of the AWS Regions where you plan to transfer data.
Copy the following
create-taskcommand:aws datasync create-task \ --source-location-arn "arn:aws:datasync:us-east-1:account-id:location/location-id" \ --destination-location-arn "arn:aws:datasync:us-east-1:account-id:location/location-id" \ --task-mode "ENHANCED-or-BASIC"For
--source-location-arn, specify the Amazon Resource Name (ARN) of your source location.For
--destination-location-arn, specify the ARN of your destination location.If you're transferring across AWS Regions or accounts, make sure that the ARN includes the other Region or account ID.
For
--task-mode, specifyENHANCEDorBASIC.For more information, seeUnderstanding task mode differences.
Specify other task options as needed. You might be interested in some of the following options:
Configure how tohandle file metadata andverify data integrity.
Monitor your transfer withtask reports orAmazon CloudWatch Logs.
For more options, seecreate-task. Here's an example
create-taskcommand that specifies Enhanced mode and several other options:aws datasync create-task \ --source-location-arn "arn:aws:datasync:us-east-1:account-id:location/location-id" \ --destination-location-arn "arn:aws:datasync:us-east-1:account-id:location/location-id" \ --name "task-name" \ --task-mode "ENHANCED" \ --options TransferMode=CHANGED,VerifyMode=ONLY_FILES_TRANSFERRED,ObjectTags=PRESERVE,LogLevel=TRANSFERRun the
create-taskcommand.If the command is successful, you get a response that shows you the ARN of the task that you created. For example:
{ "TaskArn": "arn:aws:datasync:us-east-1:111222333444:task/task-08de6e6697796f026" }
You can specify the DataSync task mode by configuring theTaskMode parameter in theCreateTask operation.
