feat(storage): setting idempotency token header by shubhangi-google · Pull Request #27338 · googleapis/google-api-ruby-client

shubhangi-google · 2026-07-17T08:28:53Z

This pull request introduces support for sending an idempotency token header (X-Goog-Gcs-Idempotency-Token) in API commands

cpriti-os

AI review: there are two issues.

Global Default Scope Leak (Critical Issue):
By setting RequestOptions.default.add_idempotency_token_header = true in google/apis/options.rb, this PR enables the GCS-specific X-Goog-Gcs-Idempotency-Token header globally for all 400+ Google API clients using google-apis-core (e.g., YouTube, Drive, Compute Engine, Calendar, Bigtable). This violates service abstraction boundaries and adds a 36-byte UUID overhead and confusing headers to every non-GCS request.
Fix: Default add_idempotency_token_header to false in google-apis-core and explicitly enable it in the google-apis-storage_v1 client generation or configuration.
Redundant Memoization:
In set_idempotency_token_header, the @idempotency_token ||= SecureRandom.uuid memoization is functionally dead code because of the early return on the line right before it:

return if header.any? { |k, _| k.to_s.downcase == 'x-goog-gcs-idempotency-token' }
@idempotency_token ||= SecureRandom.uuid

Because the header Hash is mutated in place and retained across retries of the ApiCommand instance, the first network attempt injects the header into header. On subsequent retries, the loop evaluates return if header.any? { ... }, successfully finding the original key, and returning early. Therefore, @idempotency_token is never evaluated or used again across retries.
Fix: You can simply do header['X-Goog-Gcs-Idempotency-Token'] = SecureRandom.uuid, as caching the UUID as an instance variable is unnecessary.

bajajneha27 · 2026-09-21T09:55:28Z

+
        def invocation_id_header
-          "gccl-invocation-id/#{SecureRandom.uuid}"
+          @invocation_id ||= SecureRandom.uuid


Why are we making this an instance variable?

without an instance variable, every retry would generate a brand new UUID. We use an instance variable to memoize the UUID.

As discussed offline, please do the required testing.

hi @bajajneha27 we already have test to check the Invocation ID across retries
https://github.com/googleapis/google-api-ruby-client/blob/main/google-apis-core/spec/google/apis/core/api_command_spec.rb#L249
we did the testing ,the functionality is working fine with instance variable

bajajneha27 · 2026-09-21T10:06:37Z


+        def set_idempotency_token_header
+          return if options&.header&.any? { |k, _| k.to_s.downcase == 'x-goog-gcs-idempotency-token' }
+          return if header.any? { |k, _| k.to_s.downcase == 'x-goog-gcs-idempotency-token' }


Can you please help me understand how header is populated?

When a request is made for the first time, this hash doesn't have the idempotency token, so the code populates it with a SecureRandom.uuid.
However, if the API call fails (e.g., due to a 503 Service Unavailable) and the library automatically retries the request, the prepare! method is called again. During this retry, the internal header hash still contains the UUID that was generated on the first attempt. Above line detects this and returns early, ensuring the exact same UUID is sent again on the retry.

My question is how we're populating header hash ?

The header hash is an instance attribute (accessor for @Header) of the command object, which is inherited from the base HttpCommand class. It gets initialized (typically as an empty hash) when the command object is first instantiated.

Throughout the lifecycle of the request, the prepare! method and its helpers (like set_api_client_header, set_user_project_header, and set_idempotency_token_header) mutate this exact same instance variable. Because the library reuses the same command object instance during a retry, the header hash retains the values (like the UUID) that were populated into it during the initial request's prepare! call.

bajajneha27 · 2026-09-21T10:07:22Z

    RequestOptions.default.quota_project = nil
    RequestOptions.default.add_invocation_id_header = false
    RequestOptions.default.upload_chunk_size = 100 * 1024 * 1024 # 100 MB
+    RequestOptions.default.add_idempotency_token_header = false


Is adding idempotency token going to be optional?

I thought it was going to be added by default.

as per this comment #27338 (review)
It has to be optional and default to false in core library, It is being enabled by default inside the Storage API client specifically.

My question is, do we need this option at all?
Isn't the requirement to always keep the header?

x-goog-gcs-idempotency-token is being used for storage operations specifically (as the header name contained x-goog-gcs) we don't need to keep it optional and we make it default as it will be ignored by other libraries (e.g., YouTube, Drive, Compute Engine, Calendar) which are using core library

bajajneha27 · 2026-09-21T10:12:23Z

+      command.options.header = { 'x-goog-gcs-idempotency-token' => 'my-custom-token' }
+      command.prepare!
+      expect(command.header['x-goog-gcs-idempotency-token']).to eql 'my-custom-token'
+      expect(command.header['X-Goog-Gcs-Idempotency-Token']).to be_nil


Why would it be nil here?

expect(command.header['x-goog-gcs-idempotency-token']).to eql 'my-custom-token'
Confirms that the user's custom lowercase header ('x-goog-gcs-idempotency-token') was preserved with the custom value.
expect(command.header['X-Goog-Gcs-Idempotency-Token']).to be_nil Confirms that the library didn't accidentally fall back to its default behavior and append a second, PascalCased header ('X-Goog-Gcs-Idempotency-Token'). If this were not nil, the final HTTP request would contain two conflicting idempotency headers, which could cause errors on the backend server.

shubhangi-google · 2026-09-25T10:19:09Z

AI review: there are two issues.

Global Default Scope Leak (Critical Issue):
By setting RequestOptions.default.add_idempotency_token_header = true in google/apis/options.rb, this PR enables the GCS-specific X-Goog-Gcs-Idempotency-Token header globally for all 400+ Google API clients using google-apis-core (e.g., YouTube, Drive, Compute Engine, Calendar, Bigtable). This violates service abstraction boundaries and adds a 36-byte UUID overhead and confusing headers to every non-GCS request.
Fix: Default add_idempotency_token_header to false in google-apis-core and explicitly enable it in the google-apis-storage_v1 client generation or configuration.

Redundant Memoization:
In set_idempotency_token_header, the @idempotency_token ||= SecureRandom.uuid memoization is functionally dead code because of the early return on the line right before it:
return if header.any? { |k, _| k.to_s.downcase == 'x-goog-gcs-idempotency-token' }
@idempotency_token ||= SecureRandom.uuid
Because the header Hash is mutated in place and retained across retries of the ApiCommand instance, the first network attempt injects the header into header. On subsequent retries, the loop evaluates return if header.any? { ... }, successfully finding the original key, and returning early. Therefore, @idempotency_token is never evaluated or used again across retries. Fix: You can simply do header['X-Goog-Gcs-Idempotency-Token'] = SecureRandom.uuid, as caching the UUID as an instance variable is unnecessary.

hi @cpriti-os x-goog-gcs-idempotency-token is being used for storage operations specifically (as the header name contained x-goog-gcs) we don't need to keep it optional and we make it default as it will be ignored by other libraries (e.g., YouTube, Drive, Compute Engine, Calendar) which are using core library

shubhangi-google added 15 commits April 16, 2025 12:35

setting idempotency header

c553b86

setting new token value

2f5a45a

fixing spec

a7a03f4

Merge branch 'main' into add_idempotency_token

1940188

checking token in retry case

b53b80d

updating token value

cd1a711

updating test case

e8f611a

removing not needed code

f032d76

Merge branch 'main' into add_idempotency_token

c59ec25

removing idempotency token header from options

a6db642

wip

f34f24c

adding lazzy load

af8dd7d

additional changes

fa82104

Merge branch 'main' into add_idempotency_token_modified

67e3088

Merge branch 'main' into add_idempotency_token_modified

3c6fa07

shubhangi-google mentioned this pull request Jul 17, 2026

feat(storage): Support for idempotency_token_header googleapis/google-cloud-ruby#34956

Closed

shubhangi-google marked this pull request as ready for review July 17, 2026 10:58

shubhangi-google requested a review from a team as a code owner July 17, 2026 10:58

cpriti-os suggested changes Sep 11, 2026

View reviewed changes

shubhangi-google added 3 commits September 15, 2026 12:39

Merge branch 'main' into add_idempotency_token_modified

dae7d41

removing redundant Memoization

ece2fd9

updating test

25612b9

shubhangi-google marked this pull request as draft September 16, 2026 06:01

fixing tests

6a92844

shubhangi-google marked this pull request as ready for review September 18, 2026 06:54

shubhangi-google requested a review from cpriti-os September 18, 2026 07:07

bajajneha27 suggested changes Sep 21, 2026

View reviewed changes

making idempotency header as default

60e489d

shubhangi-google requested a review from bajajneha27 September 30, 2026 09:25

Conversation

shubhangi-google commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpriti-os left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shubhangi-google commented Sep 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shubhangi-google commented Jul 17, 2026 •

edited

Loading

cpriti-os left a comment •

edited

Loading