Sitelet https://github.com/devantler-tech/.github/pull/381
Skip to content

fix(publish): verify manifests before promoting release tags - #381

Merged
devantler merged 4 commits into
mainfrom
codex/release-promotion-371
Oct 3, 2026
Merged

devantler merged 4 commits into
mainfrom
codex/release-promotion-371

Conversation

@devantler

@devantler devantler commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

Release tags currently become visible before their manifests are signed. A deployment can select them during that window and reject the release, creating avoidable warnings and delays.

What

Add an opt-in publishing mode that signs and verifies manifests before exposing their release tags. Stable releases alone update the latest tag; existing callers keep their current behavior, and the rest of the publishing improvements remain open.

👉 After merge/promotion: adopt this mode only after the signer-identity migration in #242 is reviewed and deployed.

Part of #371

Add an opt-in staging path that signs and verifies the produced digest before publishing its version and stable latest tags. Preserve existing callers until their reviewed signer migration is ready.

Part of #371
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@github-code-quality

github-code-quality Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Code Coverage Overview

Languages: Go, C#

Go / code-coverage/test

The overall line coverage in commit 49328d1 in the codex/release-promot... branch remains at 50%, unchanged from commit 0370254 in the main branch.

C# / code-coverage/dotnet

The overall line coverage in commit 49328d1 in the codex/release-promot... branch remains at 100%, unchanged from commit 0370254 in the main branch.


Updated October 03, 2026 02:57 UTC

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 424d71ba-de03-43ce-b990-b9dc08731782

📥 Commits

Reviewing files that changed from the base of the PR and between 48602bd and 575afc2.

📒 Files selected for processing (5)
  • .github/tests/test-publish-manifests-promotion.sh
  • .github/tests/test-publish-preflight.sh
  • .github/workflows/ci.yaml
  • .github/workflows/publish-manifests.yaml
  • README.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
🧰 Additional context used
📓 Path-based instructions (3)
Source excerpt: Every `.github/tests/test-*.sh` is a test entrypoint and must have an explicit invocation in a `ci.yaml` step's `run:` block.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • .github/workflows/ci.yaml
  • .github/workflows/publish-manifests.yaml
Source excerpt: Run [zizmor](https://github.com/zizmorcore/zizmor) to scan for GitHub Actions vulnerabilities

📄 CodeRabbit inference engine (actions/CONTRIBUTING.md)

Files:

  • .github/workflows/ci.yaml
  • .github/workflows/publish-manifests.yaml
Source excerpt: Pin `` to a commit SHA.

📄 CodeRabbit inference engine (README.md)

Files:

  • README.md
🪛 ast-grep (0.45.3)
.github/tests/test-publish-manifests-promotion.sh

[warning] 91-91: A credential-bearing variable (e.g. PASSWORD, PASSWD, SECRET, TOKEN, API_KEY) is assigned a hardcoded string literal. Secrets committed to a script are exposed in source control, process listings, and shell history, and cannot be rotated without a code change. Read the value from a secrets manager or an injected environment variable at runtime instead (e.g. PASSWORD="${DB_PASSWORD:?must be set}"), and never commit the literal.
Context: GH_TOKEN=offline-fixture
Note: [CWE-798] Use of Hard-coded Credentials.

(hardcoded-password-assignment-bash)

🔇 Additional comments (5)
.github/workflows/publish-manifests.yaml (1)

31-35: LGTM!

Also applies to: 161-164, 173-187, 191-191, 201-215

.github/tests/test-publish-manifests-promotion.sh (1)

1-120: LGTM!

.github/tests/test-publish-preflight.sh (1)

114-115: LGTM!

.github/workflows/ci.yaml (1)

3177-3180: LGTM!

Also applies to: 4035-4035, 4049-4049

README.md (1)

629-630: LGTM!

Also applies to: 661-661


📝 Walkthrough

Walkthrough

The Publish Manifests workflow adds an opt-in signed-promotion mode. It validates a SHA-pinned caller identity and numeric run identifiers, pushes to a per-attempt staging tag, then signs and verifies the pushed digest before assigning the version tag. It updates latest only for stable releases in this mode. The default-off path retains the prior publication behavior. New offline tests cover workflow wiring, promotion outcomes, and failure cases, and CI runs the promotion test.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 575af

No actionable issue is identified for this opt-in publishing change. The described rollout keeps existing callers on the default behavior.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 575af

The new mode strengthens release publication without increasing publishing permissions or changing existing callers by default. No introduced security vulnerability was established. Production adoption still depends on signer-identity migration, and registry recovery and retention behavior remain incompletely demonstrated.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • observed — The publisher retains contents-read, packages-write, and id-token-write permissions. Its mutations target the selected GHCR manifests package, including staging, version, latest, and signing data. The PR does not add broader permissions or another deployment authority; downstream consumer exposure depends on trust and selection configuration not supplied here.

Security Findings and Attack Paths

  • observed — The default branch still exposes release tags before signing, as the base workflow did. Signed mode removes that ordering window for its release tags. The legacy condition is unchanged rather than an introduced or worsened finding.

Trust Boundaries and Controls

  • observed — The signer reference comes from GitHub's OIDC response rather than a caller input. Signed mode requires its immutable SHA form and numeric run identifiers before the artifact push. It then binds verification to the expected identity, issuer, and digest, and promotes by digest rather than trusting a mutable staging or version alias.

Resilience and Maintainability Implications

  • observed — The offline harness asserts that push, invalid-digest, signing, verification, and version-command failures preserve existing release tags. It also checks stable and prerelease behavior and rejects malformed identities before registry calls. Its mocks do not establish real registry partial-write behavior or recovery after latest succeeds or fails.
  • observed — Staging artifacts intentionally remain after signing or verification failure. The inspected publishing workflow has no staging cleanup operation; whether an external retention process owns eventual removal remains unknown.

Hardening Proposals

  • proposed — Before consumer adoption, confirm the deployed signer identity against consumer trust rules and define ownership for staging retention and partial-promotion reconciliation. Validate interrupted and concurrent recovery without assuming that blindly rolling back a mutable tag is safe.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files. (3 skipped: 3 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: verifying manifests before promoting release tags.
Description check ✅ Passed The description directly explains the opt-in signed-promotion mode, stable-release behavior, compatibility with existing callers, and rollout dependency.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files. (3 skipped: 3 unsupported.)

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

At head 575afc2, the completed CodeRabbit review reports no actionable comments, with two ancillary warnings assessed as follows:

  • The docstring coverage warning reports three unsupported functions and three skipped analyses. These are Bash fixture helpers, not functions with missing supported docstrings; the script documents its purpose and the production-boundary assertions. There is no demonstrated missing documentation defect to fix.
  • The hardcoded-credential warning refers to GH_TOKEN=offline-fixture. This is deliberately nonfunctional fixture data passed to an extracted workflow step whose registry and signer commands are replaced by local test stubs. It is not an actual credential, does not authenticate any service, and the fixture makes no external registry or signing request.

body_findings=0-resolved@575afc2

Additional user-path evidence: the checksum-verified Flux 2.9.5 CLI was run against a temporary loopback OCI registry, without Docker or production credentials. It pushed a manifest to staging-123-2, then promoted by the returned digest. Both the version tag and latest resolved to that exact digest, and no release tag existed before promotion. This supplements the committed workflow-step regressions; it does not claim hosted keyless signing, registry failure recovery or consumer adoption has been exercised. Signed promotion remains opt-in, and adoption stays gated on #242.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Head 575afc2afad7ffbfb196216170cb6fb52b656e7d has green required readable checks in CI run 37069500008, a current-head finding-free CodeRabbit verdict, and zero unresolved threads. The code-quality rule is configured but has no readable per-head verdict; it has not been waived.

The extracted production-step tests cover the default-off and opted-in paths, including push, digest, signing, verification and promotion failures. A real Flux 2.9.5 trial against a disposable loopback OCI registry also proved that digest-based promotion preserves the produced bytes.

Blocker: the hosted keyless-signing and verification path has not been exercised at this head. Neither offline signer stubs nor the local Flux trial prove that the GitHub OIDC identity matches the issued certificate in the enabled publication path. This PR therefore remains draft rather than treating green CI as complete user-path evidence. A SHA-pinned signing trial against a disposable registry is the next delivery gate; deployment adoption also remains gated on #242.

No release workflow was activated, production registry tag changed, or signature check relaxed as part of these tests.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Fresh readback confirms the problem this PR addresses is still observable. At 2026-10-03T00:14:42Z, the deployment source rejected a newly visible artifact because its signature was not yet available. At 00:14:47Z, the same source verified the signature and reported Ready. There is no persistent source failure to bypass, and signature validation remains enabled.

This strengthens the case for signing and verifying before promoting release tags, rather than silencing the resulting log error. It does not replace the missing hosted keyless-signing trial at 575afc2afad7ffbfb196216170cb6fb52b656e7d, nor the signer-identity migration/adoption gate in #242. The existing blocker remains explicit; no production registry tag or runtime resource was changed.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

The missing hosted signing trial passed for the unchanged production workflow body at 46ab83244d3a072300c0146ed24511b931085faf (workflow blob 130d62d143b181f032e54d29128f539abc180b83). Native run 37088900547 used the shipped Flux and cosign action pins, real GitHub OIDC and a disposable TLS registry.

Stable publication signed and verified the produced digest before exposing the version and latest tags. A distinct prerelease digest became available at its version tag while latest retained the stable digest. The negative control substituted a different valid SHA in the expected certificate identity: verification failed, the release version stayed absent, and latest stayed unchanged. All three checks passed with real tools; registry bytes were checked independently through HTTP.

The temporary callee was immutable at 17adc806416ff5710fe4777a7cc4db8111914da6 and invoked by signed Go-template commit b59e06e257350b63cf05237e42337ae0f02ba9f4. Its certificate names the disposable trial workflow. This establishes the SHA-bound OIDC resolver/signing/promotion algorithm, not production workflow identity adoption, GHCR authentication or the broader immutable-version/app-publishing criteria in #371. Adoption remains gated on #242, and signed promotion remains opt-in. The temporary evaluation refs will be removed after readback.

Current-head CI 37087927674 is fully green. A new current-head review is requested because the base update invalidated the prior review head.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-03T03:05:22.177534Z 49328d1 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46ab83244d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/publish-manifests.yaml Outdated
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: a0e0dd4a90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@devantler
devantler marked this pull request as ready for review October 3, 2026 02:37
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 49328d12df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Readiness at 49328d1: the fresh main merge preserves publication workflow blob 9211e63212a7c7321c294bbef06aa750fc1d7e1c. All current-head checks are settled green, the review thread is resolved, and Codex gave a clean verdict on this exact head.

Native trial 37091513823 checked out this exact head through immutable callee cf35068. Real GitHub OIDC, Flux and cosign passed stable publication, prerelease publication with latest retained, and a wrong-identity negative control with no release-tag promotion. Independent HTTP readback matched the produced digest.

This proves the signed-promotion algorithm against a disposable TLS registry. Production identity adoption, registry authentication and the remaining #371 criteria remain separate; the feature stays opt-in and #242 remains open.

@devantler
devantler merged commit 3de2bbe into main Oct 3, 2026
270 checks passed
@devantler
devantler deleted the codex/release-promotion-371 branch October 3, 2026 03:09
devantler added a commit to devantler-tech/actions that referenced this pull request Oct 3, 2026
* fix(publish): verify manifests before promoting release tags

Backport the default-off signed promotion from devantler-tech/.github#381 for unmigrated consumers. Reproduced the unsigned release-tag exposure before implementation; stage under a non-semver attempt tag, sign and verify the produced digest, then promote that digest. Preserve the legacy default and signing identity.

Part of #1395

* test(publish): cover sequential latest promotion failure

Keep the verified version when the subsequent latest tag update fails; do not imply registry-wide atomicity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant