- Notifications
You must be signed in to change notification settings - Fork166
Closed
Assignees
Description
I'm seeing frequent 503 and 504 responses using theblob.upload_from_filename() method. An example trace looks like this:
503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: { "error": { "code": 503, "message": "Backend Error", "errors": [ { "message": "Backend Error", "domain": "global", "reason": "backendError" } ] }}: (u'Request failed with status code', 503, u'Expected one of', 200)' - Calling rollback() ...Traceback (most recent call last): File "/my/app/module.py", line 112, in upload blob.upload_from_filename(file_path) File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2471, in upload_from_filename checksum=checksum, File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 2359, in upload_from_file _raise_from_invalid_response(exc) File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/blob.py", line 3886, in _raise_from_invalid_response raise exceptions.from_http_status(response.status_code, message, response=response)ServiceUnavailable: 503 POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=multipart: { "error": { "code": 503, "message": "Backend Error", "errors": [ { "message": "Backend Error", "domain": "global", "reason": "backendError" } ] }}: (u'Request failed with status code', 503, u'Expected one of', 200)It looks like this method callsblob.upload_from_file() withnum_retries=None, so there is no retry strategy applied. I also see that thenum_retries argument is documented as deprecated.
Does this library provide an alternative way to retryblob.upload_from_filename() orblob.upload_from_file()? I can wrap these calls in retry logic of my own, but I see that retries are enabled by default for other functions.
I'm usinggoogle-cloud-storage==1.34.0. I've already pulled ingoogle-resumable-media==1.20.0 so I should be getting the recent resumable media retry fixes that came with#342.
Thanks for any tips on retrying uploads.