auth: token cache entry written with null expires_at is never refreshed, causing permanent 401 ACCESS_TOKEN_TYPE_UNSUPPORTED for one scope #904

Summary

A token_cache.json entry can be written with expires_at: None. Because gws can then never conclude the entry is stale, it never refreshes it and keeps re-sending the same access token indefinitely. Once that token's grant ages out or is superseded, every call needing that scope fails permanently with:

401 UNAUTHENTICATED
"Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential."
details[].reason = ACCESS_TOKEN_TYPE_UNSUPPORTED

It does not self-recover, and gws auth login does not clear the cache, so re-authenticating does not fix it either. In my case gws gmail +watch was broken for ~5 days across two successful re-auths, failing at its Pub/Sub topic-creation step, while every Gmail call kept working normally.

Version / environment

• gws 0.22.5 (current latest release)
• Linux (Debian 12) in Docker, headless, keyring_backend: keyring
• OAuth user credentials (credentials.enc), 27 scopes granted including pubsub and cloud-platform

Evidence

I preserved the bad token_cache.json and decrypted it (AES-256-GCM, 12-byte nonce prefix, key from .encryption_key). It decrypts to a dict keyed by scope URL, each value {access_token, refresh_token, expires_at, id_token}.

Every entry had a real expires_at except the one for the failing scope:

key                            access_token     expires_at
-----------------------------  ---------------  ------------------------------
.../auth/pubsub                ya29… (254 ch)   None            <-- never refreshed
.../auth/gmail.modify          ya29… (254 ch)   [2026, 233, 6, 29, 26, …]
https://mail.google.com/       ya29… (254 ch)   [2026, 233, 6, 34, 39, …]
.../auth/chat.bot              ya29… (253 ch)   [2026, 232, 16, 14, 46, …]
.../auth/chat.admin.spaces     ya29… (253 ch)   [2026, 232, 16, 34, 22, …]

The cached access_token was a well-formed ya29. value, not corrupt data — just long dead. ACCESS_TOKEN_TYPE_UNSUPPORTED appears to be what the API frontend returns for a bearer token it cannot resolve at all; I confirmed that a garbage string, a truncated ya29.…, and a 1//…-shaped value all produce that same reason, whereas an unauthenticated call gives 403 PERMISSION_DENIED instead. So the reason code is misleading here — it reads like "wrong credential type" but the real condition is "unresolvable token".

Deleting token_cache.json fixes it immediately and permanently.

Why this is easy to misdiagnose

Two properties made this cost several days of investigation:

1. The cache is per-scope, so the failure is per-API. Gmail calls kept succeeding off their own valid entry, so gws auth status was clean, gws gmail users messages list worked, and auth looked entirely healthy. Only the one scope was dead.
2. The error is byte-identical across credential changes, because the credential is never consulted — the cache short-circuits it. I replaced the credential twice and re-ran gws auth login twice with identical output each time, which strongly (and wrongly) suggested the problem was outside gws.

There was also a red herring worth mentioning: the same command worked on a desktop machine with the same account, binary and project, and failed on the server, which looked like an environment difference. It wasn't. Per-scope entries are created lazily on first use, so the desktop run minted a fresh entry and used it within the hour; the server had one cached from days earlier that could never refresh. Same latent bug on both machines — it only bites on the second and later runs.

Reproduction

Non-destructive A/B, using GOOGLE_WORKSPACE_CLI_CONFIG_DIR against a copy of the config dir:

mkdir -p /tmp/gwscfg && chmod 700 /tmp/gwscfg
cp -a ~/.config/gws/client_secret.json ~/.config/gws/credentials.enc \
      ~/.config/gws/.encryption_key /tmp/gwscfg/

# no token_cache.json present -> succeeds
GOOGLE_WORKSPACE_CLI_CONFIG_DIR=/tmp/gwscfg gws gmail +watch --project <PROJECT> \
  --label-ids INBOX --once --cleanup --output-dir inbox

# same dir, with the bad token_cache.json copied in -> 401 ACCESS_TOKEN_TYPE_UNSUPPORTED
cp -a ~/.config/gws/token_cache.json /tmp/gwscfg/
GOOGLE_WORKSPACE_CLI_CONFIG_DIR=/tmp/gwscfg gws gmail +watch --project <PROJECT> \
  --label-ids INBOX --once --cleanup --output-dir inbox

I don't have a deterministic reproducer for how the null expires_at gets written in the first place — that's the part I can't pin down without the auth internals. The entry was created by gmail +watch's own per-scope token acquisition (gmail and pubsub scopes are fetched separately there). It may be relevant that this happened on a machine where the credential had previously gone bad with invalid_grant: Token has been expired or revoked before being re-authed.

Expected behaviour

Any one of these would have prevented a multi-day outage:

1. Never persist a cache entry with an unknown expiry. If the token response has no usable expiry, either don't cache it or treat a missing expires_at as "already expired" so the next call refreshes. A null expiry currently means "valid forever", which is the opposite of the safe default.
2. Invalidate the cache entry on a 401 from the API. There is currently no negative feedback: a token that the server rejects stays cached and is resent forever. One retry-after-refresh on UNAUTHENTICATED would make this self-healing.
3. Clear (or scope-invalidate) token_cache.json on gws auth login. After a successful login the cached tokens belong to a superseded grant — and the new login may carry a different scope set or even a different account. On my machine credentials.enc was rewritten at 08:22 while token_cache.json remained untouched from 08:05.

(3) is independently worth fixing: as it stands, "re-authenticate" is not a reliable remedy for any token-level problem, which is counter-intuitive for anyone debugging auth.

Possibly related: #205

#205 ("Apps Script API returns 401 … while Drive API works with same credentials") looks like the same bug observed from the outside: same 401 message, one API dead while another works off the same credential — which is the per-scope cache signature. Notably, a commenter there reports that deleting credentials*.enc and token_cache*.json and re-authorising resolves it, which matches what I found. That issue is on 0.4.4 and covers a couple of other problems too, so I've filed this separately with the mechanism and the decrypted evidence rather than adding to it — but they may well share a cause, and if so fixing the expiry/invalidation handling would close both.

Workaround

rm ~/.config/gws/token_cache.json — gws regenerates it from the stored credential on the next call, and nothing is lost. Worth noting in the docs, since re-auth alone does not do it.