When an application transfers data using the Azure Storage client library for JavaScript, there are several factors that can affect speed, memory usage, and even the success or failure of the request. To maximize performance and reliability for data transfers, it's important to be proactive in configuring client library transfer options based on the environment your app runs in.

This article walks through several considerations for tuning data transfer options. When properly tuned, the client library can efficiently distribute data across multiple requests, which can result in improved operation speed, memory usage, and network stability.

Performance tuning for uploads

Properly tuning data transfer options is key to reliable performance for uploads. Storage transfers are partitioned into several subtransfers based on the values of these arguments. The maximum supported transfer size varies by operation and service version, so be sure to check the documentation to determine the limits. For more information on transfer size limits for Blob storage, seeScale targets for Blob storage.

Set transfer options for uploads

You can configure properties inBlockBlobParallelUploadOptions to improve performance for data transfer operations. The following table lists the properties you can configure, along with a description:

maxSingleShotSize

ThemaxSingleShotSize value is the maximum blob size in bytes for a single request upload.

If the size of the data is less than or equal tomaxSingleShotSize, the blob is uploaded with a singlePut Blob request. If the blob size is greater thanmaxSingleShotSize, or if the blob size is unknown, the blob is uploaded in chunks using a series ofPut Block calls followed byPut Block List.

It's important to note that the value you specify forblockSizedoes not limit the value that you define formaxSingleShotSize. ThemaxSingleShotSize argument defines a separate size limitation for a request to perform the entire operation at once, with no subtransfers. It's often the case that you wantmaxSingleShotSize to beat least as large as the value you define forblockSize, if not larger. Depending on the size of the data transfer, this approach can be more performant, as the transfer is completed with a single request and avoids the overhead of multiple requests.

If you're unsure of what value is best for your situation, a safe option is to setmaxSingleShotSize to the same value used forblockSize.

blockSize

TheblockSize value is the maximum length of a transfer in bytes when uploading a block blob in chunks.

As mentioned earlier, this valuedoes not limitmaxSingleShotSize, which can be larger thanblockSize.

To keep data moving efficiently, the client libraries might not always reach theblockSize value for every transfer. Depending on the operation, the maximum supported value for transfer size can vary. For more information on transfer size limits for Blob storage, see the chart inScale targets for Blob storage.

Code example

The following code example shows how to set values forBlockBlobParallelUploadOptions and include the options as part of an upload method call. The values provided in the samples aren't intended to be a recommendation. To properly tune these values, you need to consider the specific needs of your app.

// Specify data transfer optionsconst uploadOptions = {  blockSize: 4 * 1024 * 1024, // 4 MiB max block size  concurrency: 2, // maximum number of parallel transfer workers  maxSingleShotSize: 8 * 1024 * 1024, // 8 MiB initial transfer size} // Create blob client from container clientconst blockBlobClient = containerClient.getBlockBlobClient(blobName);// Upload blob with transfer optionsawait blockBlobClient.uploadFile(localFilePath, uploadOptions);

// Specify data transfer optionsconst uploadOptions: BlockBlobParallelUploadOptions = {  blockSize: 4 * 1024 * 1024, // 4 MiB max block size  concurrency: 2, // maximum number of parallel transfer workers  maxSingleShotSize: 8 * 1024 * 1024, // 8 MiB initial transfer size};// Create blob client from container clientconst blockBlobClient: BlockBlobClient = containerClient.getBlockBlobClient(blobName);await blockBlobClient.uploadFile(localFilePath, uploadOptions);

In this example, we set the maximum number of parallel transfer workers to 2 using theconcurrency property. We also setmaxSingleShotSize to 8 MiB. If the blob size is smaller than 8 MiB, only a single request is necessary to complete the upload operation. If the blob size is larger than 8 MiB, the blob is uploaded in chunks with a maximum chunk size of 4 MiB, which we define in theblockSize property.

Performance considerations for uploads

During an upload, the Storage client libraries split a given upload stream into multiple subuploads based on the configuration options defined byBlockBlobParallelUploadOptions. Each subupload has its own dedicated call to the REST operation. In this example, the operation isPut Block. The Storage client library manages these REST operations in parallel (depending on transfer options) to complete the full upload.

Buffering during uploads

The Storage REST layer doesn’t support picking up a REST upload operation where you left off; individual transfers are either completed or lost. To ensure resiliency for stream uploads, the Storage client libraries buffer data for each individual REST call before starting the upload. In addition to network speed limitations, this buffering behavior is a reason to consider a smaller value forblockSize, even when uploading in sequence. Decreasing the value ofblockSize decreases the maximum amount of data that is buffered on each request and each retry of a failed request. If you're experiencing frequent timeouts during data transfers of a certain size, reducing the value ofblockSize reduces the buffering time, and might result in better performance.

Performance tuning for downloads

Tuning data transfer options for downloads is available only when using thedownloadToBuffer method. This method downloads a blob in parallel to a buffer based on the values defined inBlobDownloadToBufferOptions. Other download methods don't support tuning data transfer options.

Set transfer options for downloads

The following values can be tuned for downloads when using thedownloadToBuffer method:

• blockSize: The maximum block size to transfer for each request.
• concurrency: The maximum number of parallel requests issued at any given time as a part of a single parallel transfer.

Performance considerations for downloads

During a download usingdownloadToBuffer, the Storage client libraries split a given download request into multiple subdownloads based on the configuration options defined byBlobDownloadToBufferOptions. Each subdownload has its own dedicated call to the REST operation. Depending on transfer options, the client libraries manage these REST operations in parallel to complete the full download.

Code example

The following code example shows how to set values forBlobDownloadToBufferOptions and include the options as part of adownloadToBuffer method call. The values provided in the samples aren't intended to be a recommendation. To properly tune these values, you need to consider the specific needs of your app.

// Specify data transfer optionsconst downloadToBufferOptions = {    blockSize: 4 * 1024 * 1024, // 4 MiB max block size    concurrency: 2, // maximum number of parallel transfer workers}// Download data to bufferconst result = await client.downloadToBuffer(offset, count, downloadToBufferOptions);

// Specify data transfer optionsconst downloadToBufferOptions: BlobDownloadToBufferOptions = {    blockSize: 4 * 1024 * 1024, // 4 MiB max block size    concurrency: 2, // maximum number of parallel transfer workers}// Download data to bufferconst result = await client.downloadToBuffer(offset, count, downloadToBufferOptions);

Related content

• To understand more about factors that can influence performance for Azure Storage operations, seeLatency in Blob storage.
• To see a list of design considerations to optimize performance for apps using Blob storage, seePerformance and scalability checklist for Blob storage.