Resumable uploads

Setup

This page discusses resumable uploads in Cloud Storage. Resumable uploads arethe recommended method for uploading large files, because you don't have torestart them from the beginning if there is a network failure while the uploadis underway.

Introduction

Aresumable upload lets you resume data transfer operations toCloud Storage after a communication failure has interrupted the flowof data. Resumable uploads work by sending multiple requests, each of whichcontains a portion of the object you're uploading. This is different from asingle-request upload, which contains all of the object's data in a singlerequest and must restart from the beginning if it fails part way through.

  • Use a resumable upload if you are uploading large files or uploading over aslow connection. For example file size cutoffs for using resumable uploads,seeupload size considerations.

  • A resumable upload must be completed within a week of being initiated, but canbecancelled at any time.

  • Only a completed resumable upload appears in your bucket and, if applicable,replaces an existing object with the same name.

    • The creation time for the object is based on when the upload completes.

    • Object metadata set by the user is specified in the initial request.This metadata is applied to the object once the upload completes.

    • The JSON API also supports settingcustom metadata in the finalrequest if you include headers prefixed withX-Goog-Meta- in thatrequest.

How tools and APIs use resumable uploads

Depending on how you interact with Cloud Storage, resumable uploadsmight be managed automatically on your behalf. This section describes theresumable upload behavior for different tools and provides guidance onconfiguring the appropriate buffer size for your application.

Console

The Google Cloud console manages resumable uploads automatically on yourbehalf. However, if you refresh or navigate away from theGoogle Cloud console while an upload is underway, the upload is cancelled.

Command line

The gcloud CLI uses resumable uploads in thegcloud storage cp andgcloud storage rsync commands whenuploading data to Cloud Storage. If your upload is interrupted,you can resume it by running the same command that you used to start theupload. When resuming such an upload that includes multiple files, usethe--no-clobber flag to prevent re-uploading files that alreadycompleted successfully.

Client libraries

When performing resumable uploads, client libraries function aswrappers around the Cloud Storage JSON API.

C++

Functions instorage::Client perform with different behavior:

By default,UploadFile() performs a resumable upload when the objectis larger than 20 MiB. Otherwise, it performs a simple upload ormultipart upload. You can configure this threshold by settingMaximumSimpleUploadsSizeOption when creating astorage::Client.

8 MiB is the default buffer size, which you canmodify with theUploadBufferSizeOption option.

The C++ client library uses a buffer size that's equal to the chunksize. The buffer size must be a multiple of 256 KiB (256 x 1024 bytes).When usingWriteObject() andUploadFile(), you might want toconsider the tradeoffs between upload speed and memory usage. Usingsmall buffers to upload large objects can make the upload slow. For moreinformation on the relationship between upload speed and buffer size forC++, see thedetailed analysis in GitHub.

C#

When uploading, the C# client library always performs resumable uploads.You can initiate a resumable upload withCreateObjectUploader.

The C# client library uses a buffer size that's equal to the chunk size.The default buffer size is 10 MB and you can change this value bysettingChunkSize onUploadObjectOptions. Thebuffer size must be a multiple of 256 KiB (256 x 1024 bytes). Largerbuffer sizes typically make uploads faster, but note that there's atradeoff between speed and memory usage.

Go

By default, resumable uploads occur automatically when the file islarger than 16 MiB. You change the cutoff for performing resumableuploads withWriter.ChunkSize. Resumable uploads arealways chunked when using the Go client library.

Multipart uploads occur when the object is smaller thanWriter.ChunkSize or whenWriter.ChunkSize is set to 0, wherechunking becomes disabled. TheWriter isunable to retry requests ifChunkSize is set to 0.

The Go client library uses a buffer size that's equal to the chunk size.The buffer size must be a multiple of 256 KiB (256 x 1024 bytes). Largerbuffer sizes typically make uploads faster, but note that there's atradeoff between speed and memory usage. If you're running severalresumable uploads concurrently, you should setWriter.ChunkSize to avalue that's smaller than 16 MiB to avoid memory bloat.

Note that the object is not finalized in Cloud Storage untilyou callWriter.Close() and receive a successresponse.Writer.Close returns an error if the request isn'tsuccessful.

Java

The Java client library has separate methods for multipart and resumable uploads. Thefollowing methods always perform a resumable upload:

The default buffer size is 15 MiB. You can set the buffer size either byusing theWriteChannel#setChunkSize(int) method, orby passing in abufferSize parameter to theStorage#createFrom method. The buffer size has ahard minimum of 256KiB. When callingWriteChannel#setChunkSize(int) internally, thebuffer size is shifted to a multiple of 256 KiB.

Buffering for resumable uploads functions as a minimum flush threshold,where writes smaller than the buffer size are buffered until a writepushes the number of buffered bytes above the buffer size.

If uploading smaller amounts of data, consider usingStorage#create(BlobInfo, byte[], Storage.BlobTargetOption...)orStorage#create(BlobInfo, byte[], int, int, Storage.BlobTargetOption...).

Node.js

Resumable uploads occur automatically. You can turn off resumableuploads by settingresumable onUploadOptions tofalse. Resumable uploads are automatically managed when using thecreateWriteStream method.

There is no default buffer size andchunked uploads must bemanually invoked by setting thechunkSize option onCreateResumableUploadOptions. IfchunkSize isspecified, the data is sent in separate HTTP requests, each with apayload of sizechunkSize. If nochunkSize is specified and thelibrary is performing a resumable upload, all data is streamed into asingle HTTP request.

The Node.js client library uses a buffer size that's equal to the chunksize. The buffer size must be a multiple of 256 KiB (256 x 1024 bytes).Larger buffer sizes typically make uploads faster, but note that there'sa tradeoff between speed and memory usage.

PHP

By default, resumable uploads occur automatically when the object sizeis larger than 5 MB. Otherwise, multipart uploads occur. This thresholdcannot be changed. You can force a resumable upload by setting theresumable option in theupload function.

The PHP client library uses a buffer size that's equal to the chunksize.256KiB is the default buffer size for a resumable upload,and you can change the buffer size by setting thechunkSize property.The buffer size must be a multiple of 256 KiB (256 x 1024 bytes). Largerbuffer sizes typically make uploads faster, but note that there's atradeoff between speed and memory usage.

Python

Resumable uploads occur when the object is larger than8 MiB,and multipart uploads occur when the object is smaller than 8 MiBThis threshold cannot be changed. The Python client library uses abuffer size that's equal to the chunk size. 100 MiB is thedefault buffer size used for a resumable upload, and youcan change the buffer size by setting theblob.chunk_size property.

To always perform a resumable uploadregardless of object size, use the classstorage.BlobWriter or the methodstorage.Blob.open(mode='w'). For these methods, thedefault buffer size is 40 MiB. You can also useResumable Media tomanage resumable uploads.

The chunk size must be a multiple of 256 KiB (256 x 1024 bytes). Largerchunk sizes typically make uploads faster,but note that there's a tradeoff between speed and memory usage.

Ruby

The Ruby client library treats all uploads as non-chunked resumableuploads.

REST APIs

JSON API

The Cloud Storage JSON API uses aPOST Object request thatincludes the query parameteruploadType=resumable to initiate theresumable upload. This request returns assession URI that youthen use in one or morePUT Object requests to upload the object data.For a step-by-step guide to building your own logic for resumableuploading, seePerforming resumable uploads.

XML API

The Cloud Storage XML API uses aPOST Object request thatincludes the headerx-goog-resumable: start to initiate theresumable upload. This request returns assession URI that youthen use in one or morePUT Object requests to upload the object data.For a step-by-step guide to building your own logic for resumableuploading, seePerforming resumable uploads.

Resumable uploads of unknown size

The resumable upload mechanism supports transfers where the file size is notknown in advance. This can be useful for cases like compressing an objecton-the-fly while uploading, since it's difficult to predict the exact file sizefor the compressed file at the start of a transfer. The mechanism is usefuleither if you want to stream a transfer that can be resumed after beinginterrupted, or if chunked transfer encoding does not work for your application.

For more information, seeStreaming uploads.

Upload performance

Choosing session regions

Resumable uploads are pinned in the region where you initiate them. Forexample, if you initiate a resumable upload in the US and give thesession URIto a client in Asia, the upload still goes through the US. To reducecross-region traffic and improve performance, you should keep a resumable uploadsession in the region in which it was created.

If you use a Compute Engine instance to initiate a resumable upload, theinstance should be in the same location as the Cloud Storage bucketyou upload to. You can then use a geo IP service to pick the Compute Engineregion to which you route customer requests, which helps keep trafficlocalized to a geo-region.

Uploading in chunks

If possible, avoid breaking a transfer into smaller chunks and instead uploadthe entire content in a single chunk. Avoiding chunking removes added latencycosts and operations charges from querying the persisted offset of each chunk aswell as improves throughput. However, you should consider uploading in chunkswhen:

  • Your source data is being generated dynamically and you want to limit howmuch of it you need to buffer client-side in case the upload fails.

  • Your clients have request size limitations, as is the case for manybrowsers.

If you're using the JSON or XML API and your client receives an error, they canquery the server for the persisted offset andresume uploading remainingbytes from that offset. The Google Cloud console, Google Cloud CLI, and clientlibraries handle this automatically on your behalf. SeeHow tools and APIs use resumable uploads for more guidance on chunking forspecific client libraries.

Considerations

This section is useful if you are building your own client that sendsresumable upload requests directly to the JSON or XML API.

Session URIs

When youinitiate a resumable upload, Cloud Storage returns asession URI, which you use in subsequent requests to upload the actual data.An example of a session URI in the JSON API is:

https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=resumable&name=my-file.jpg&upload_id=ABg5-UxlRQU75tqTINorGYDgM69mX06CzKO1NRFIMOiuTsu_mVsl3E-3uSVz65l65GYuyBuTPWWICWkinL1FWcbvvOA

An example of a session URI in the XML API is:

 https://storage.googleapis.com/my-bucket/my-file.jpg?upload_id=ABg5-UxlRQU75tqTINorGYDgM69mX06CzKO1NRFIMOiuTsu_mVsl3E-3uSVz65l65GYuyBuTPWWICWkinL1FWcbvvOA

This session URI acts as an authentication token, so the requests that use itdon't need to be signed and can be used by anyone to upload data to the targetbucket without any further authentication. Because of this, be judicious insharing the session URI and only share it over HTTPS.

A session URI expires after one week but can becancelled prior toexpiring. If you make a request using a session URI that is no longer valid,you receive one of the following errors:

  • A410 Gone status code if it's been less than a week since theupload was initiated.
  • A404 Not Found status code if it's been more than a week sincethe upload was initiated.

In both cases, or if you lose the session URI before the upload is completed,you have to initiate a new resumable upload, obtain a new session URI, and startthe upload from the beginning using the new session URI.

Integrity checks

We recommend that you request an integrity check of the final uploaded objectto be sure that it matches the source file. You can do this by calculating theMD5 digest of the source file and adding it to theContent-MD5 requestheader.

Checking the integrity of the uploaded file is particularly important if you areuploading a large file over a long period of time, because there is an increasedlikelihood of the source file being modified over the course of the uploadoperation.

However, you can't perform integrity checks on intermediate portions or chunksof a resumable upload, because resumable uploads are meant to let you resumean upload in the event it is interrupted unexpectedly.

Retries and resending data

Once Cloud Storage persists bytes in a resumable upload, those bytescannot be overwriten, and Cloud Storage ignores attempts to do so.Because of this, you should not send different data when rewinding to an offsetthat you sent previously.

For example, say you're uploading a 100,000 byte object, and your connection isinterrupted. When youcheck the status, you find that 50,000 bytes weresuccessfully uploaded and persisted. If you attempt to restart the upload atbyte 40,000, Cloud Storage ignores the bytes you send from 40,000 to50,000. Cloud Storage begins persisting the data you send at byte50,001.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-19 UTC.