storage: "Failed to parse Content-Range header" when chunk-size upload is used

Here is the log:

Caused by: com.google.gcloud.storage.StorageException: 400 Bad Request
Failed to parse Content-Range header.
at com.google.gcloud.spi.DefaultStorageRpc.translate(DefaultStorageRpc.java:94)
at com.google.gcloud.spi.DefaultStorageRpc.write(DefaultStorageRpc.java:499)
at com.google.gcloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:49)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.gcloud.RetryHelper.doRetry(RetryHelper.java:181)
at com.google.gcloud.RetryHelper.runWithRetries(RetryHelper.java:247)
at com.google.gcloud.RetryHelper.runWithRetries(RetryHelper.java:237)
at com.google.gcloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:46)
at com.google.gcloud.BaseWriteChannel.close(BaseWriteChannel.java:147)

If I uploaded a file with size is exactly 2 x 1024 x 1024, and 1024 x 1024 as the chunk size, then I'll get above exception. However, if I changed the file size to 2 x 1024 x 1024 + 1, then the problem will go away.

Apparently, there is a bug in BlobWriteChannel->flushbuffer, if there is no data available, then there is no need to call options().rpc().write.

@sudison

Hi @sudison thanks for the report. I managed to reproduce you issue, I will look into this and provide a fix as soon as possible.

PR #714 has a potential fix for that, making a note to try the repro after that request is merged.

I tried the fix in jean-philippe-martin@73b9ef6, I got

com.google.api.client.http.HttpResponseException: 400 Bad Request
{
"error": {
"errors": [
{
"domain": "global",
"reason": "badRequest",
"message": "Invalid Upload Request"
}
],
"code": 400,
"message": "Invalid Upload Request"
}
}

if length is zero, sending a upload request to cloud storage, will get above exception.

@sudison

Yes, @sudison thank you. We realized that but unfortunately only added a comment in the PR and not here. We are waiting to see if there is a good way to fix that other than applying a workaround to fix it.

My fix for this issue is to add a method:
void totalLength(long length);
on WriteChannel.
The assumption is that, caller knows the length of the file before uploading to GCS.

Indeed that is a way, but I don't think we should make knowing the total lengths a requirement (and GCS should be able to support it). There are various cases where one would not know the length and we would like to support them to.

Agreed. Having to specify the length would make Java NIO integration really hard as NIO users don't give the length before starting their writes via e.g. newOutputStream.