feat: outgoing webhooks delivered from SQS with a dead-letter queue (D9b) - #13
Conversation
Both blocks were merged by the owner on 2026-09-28, D8 first and D9a rebased onto it. With them no domain is gated any more. The task board marks P6 and X1 green, and the in-flight paragraph states the current upstream and product position, the integration PR and wave 2 that follow, and the owner's 2026-09-30 decision to freeze the parity target on @awesome-lang-auth/node 1.10.8 with a catch-up block before v1.0.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the admin console merged, `tools.auth: admin` no longer has to be
refused at cold start. toolsAccess returns
auth.ToolsProtected(core.AdminGuard(base).Protect) - the same value the
adapter guards <admin>/api/* with - so the callers are exactly the
console's, and its refusals are the console's 401/403. checkToolsSupport
and its call in New are gone; internal/config still requires
admin.enabled for the posture, and toolsAccess refuses a Config that
bypassed the loader with no console mounted.
No double-submit is layered on it: the reference's admin guard has
none, and RS-18 keeps SameSite off none under a session policy.
TestToolsAccessPostures drives the bootstrap secret (202), an anonymous
caller (401 {"error":"Unauthorized"}), an ordinary user by bearer and by
cookie (403), and toolsAccess's own refusal with the console off.
docs/config-reference.md 17.6 gains the admin posture paragraph and the
apiKey paragraph names the console as the key store's other consumer;
the SAM ToolsAuth description says why admin is not offered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reference's CORS layer is router.use inside the auth router
(auth.router.ts:512-527) and createToolsRouter emits no Access-Control
header. Mounted beside the prefix - the shape tools.router.ts:114
documents, and this product's default - the tools router never meets
that layer, so corsExemptMounts now exempts it as it exempts the admin
console. Mounted under the prefix - as the Angular demo does,
router.use('/tools', ...) on the router served at /api/auth
(ng-awesome-node-auth src/server/auth.routes.ts:98-99, src/server.ts:65)
- every request passes the auth router's CORS first, so there the
mount stays wrapped.
TestToolsMountFollowsTheReferenceCORSGeometry pins both shapes; one
sentence each in docs/config-reference.md 17.6 and 16.2. Conformance,
not a deviation, so no register entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and webhooks toolsKnobGaps reported stores.enable.apiKeys and stores.enable.webhooks as "nothing in this build reads it" whenever the tools block was off (or, for apiKeys, under a posture other than apiKey) - even with the admin console mounted, whose credential routes mint and revoke keys and manage webhook subscriptions through those same stores. A mounted console (httpConfig(cfg).AdminMounted()) now silences both rows; telemetry, which reaches a route through ToolsOptions alone, stays inert with the block off. Two rows in TestUnwiredKnobsIsExactlyTheDocumentedList pin it; the docs/config-reference.md 17 and 17.1 lists and the driverStores comment say the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…number
docs/spec/config-schema.md section 2 still had the placeholder row
"RS-14-RS-16 reserved: the tools block (D9a) numbered its rules first",
although both blocks are merged. It now carries D9a's three real rows
(RS-14 distributor, RS-15 inbound webhooks, RS-16 unset tools.auth).
internal/config/errors.go says the same: the RS-16 comment no longer
claims the rule is absent from the table, and the RS-14..16 note is in
the present tense. decisions.md has a single D-21, as required.
docs/config-reference.md had two sections numbered 17 ("Two worked
postures" and "tools.*, knob by knob"); the worked postures move to the
end as section 18. Nothing referenced the old number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ense Both blocks are on main, but several comments written on one side still spoke of the other as future work: ui.go (the console's settings routes are what can write settings.UI), internal/store/dynamodb/webhooks.go (the console's admin route defaults events and isActive), admin.go (driverStores lists telemetry because the tools block hands it over), the coreOptionSets wantEmpty comment (both remaining slots are empty on purpose, nothing is waiting), and the phase-gate tests in internal/config/phases_test.go (the empty set is the state, not a flip the parent still owes; the skipped tests say why they skip). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… costs at 2.8 The index row for templates-dir-only-seeds-absent-ids was lost when D9a was rebased onto the admin console: the register entry and its pin in deviations_test.go stayed, and TestDeviationsIndexIsComplete kept passing because the runtime-settings row cites the ID. The row is back as it stood before the merge, and the test now requires every product deviation to open an index row of its own. main's docs/cost-model.md numbers uploads 2.6, the console 2.7 and the tools block 2.8, but every tools reference still said 2.6 (two register Spec fields and their index rows, the bridge comment in tools.go, config-reference 17.2, the SAM tools banner). They now say 2.8; the uploads references stay at 2.6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…claim TestUnwiredKnobsIsExactlyTheDocumentedList covered only the tools block's rows; it now also drives the admin block's two knobs over core constants, ui.uploadDir in its filesystem spelling, and both blocks at once, so the one list the cold start logs is pinned as a union. TestTemplatesAreBackedByEveryDriver said its second half requires the early, named refusal of an unbacked store, but after the merge it only re-checked that telemetry is accepted. With every stores.enable flag handed over there is no unbacked key left, so it now pins the other side of the claim: both drivers back the same set, and an unknown driver is refused early and by name. idp.go: restore the paragraph break before the adminRL paragraph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- TestDriverStoresListTheConsoleStores no longer speaks of a flag that stays out: all six are asserted present. - TestTemplatesAreBackedByEveryDriver says the per-flag refusal of checkStoreSupport is unreachable through either driver, and asserts both drivers are known and non-empty so the symmetry loop cannot pass vacuously. - TestUnwiredDomainViaEnvIsAlsoRefused's skip message says what its comment says: it speaks again the day a domain is gated. Review findings (PR #12, conventions 5 and 9a/9c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The console is mounted, so "will expose once the admin surface is mounted" is stale, and the users note no longer names the first-user access policy, which RS-17 refuses on every driver. The notes and their byte-for-byte quotes in docs/deviations.md change together, as TestDeviationsIndexIsComplete requires. Review finding (PR #12, conventions 8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…RS gives the tools mount validateMounts refused a tools or admin mount at "/" or equal to http.apiPrefix, but not one above it (tools.basePath /api under http.apiPrefix /api/auth). corsExemptMounts exempts such a tools mount as "beside the prefix", and the exemption matches everything below the mount, so every auth route lost the CORS layer: no Allow-Origin, no Vary, no preflight answer. It is now refused with the same diagnostic shape as the prefix itself, for either knob, and pinned in TestRefuseToStartRules. Under the prefix the tools mount stays inside the layer, as the reference's geometry requires; §17.6 and the corsExemptMounts comment now say what that means under tools.auth: admin -- an allow-listed origin gets credentialed CORS on routes that take a console credential. The assembleHandler layer list and the SAM ToolsBasePath description name the tools mount and the new refusal. Review findings (PR #12, security 3 and 4, wire 4 / conventions 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p header and SAM wrap - "a tools store with the block off" now switches on stores.enable.webhooks too, so the row toolsKnobGaps reports for it with no console mounted is pinned; deleting that branch now fails the case. - The toolsKnobGaps header no longer speaks of a flag's "one consumer": the flags are consumed by the tools block and, for two of them, by the console. - The ToolsAuth description in the SAM template is re-wrapped from the over-long line to the paragraph's width; the rendered value is unchanged. Review findings (PR #12, conventions 1, 4 and 7). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n on it behind an open console, register the core's redirect
Three things about tools.auth: admin the review found unsaid or unguarded.
CSRF. Under a session policy the console's guard reads the same accessToken
cookie the session posture does, and nothing checked the double-submit in
front of it, so an admitted administrator's cookie could carry POST
<tools>/track from a same-site page. The reference's documented guard for the
tools router is auth.middleware() (tools.router.ts:114), which performs it
(auth.middleware.ts:33-41). The vendored console calls no tools route, so the
guard is now wrapped as `session` is: a cookie caller needs the matching
X-CSRF-Token, a bearer caller does not, and the cookie looked for is the one
the guard reads (<admin.cookiePrefix>accessToken when that knob is set).
The legacy guard and `open` read no cookie and are not wrapped.
Open console. Behind admin.accessPolicy: open the guard reads no credential,
so the pair is `none` by another name. It now gets `none`'s load-time warning
(collectWarnings) and cold-start Warn, and a plain Info line under every
other policy; §17.1, §17.6 and the tools.go comments no longer say "nobody
else". The `none` remedy names admin too.
Redirect. Under a session policy with admin.loginPath set, an unauthenticated
text/html request to a tools route gets the core's 302 to
<loginPath>?redirect=<admin mount><tools path>, a path nothing serves; the
reference answers 401 on every route but the panel since 1.10.0. Registered
as the core-caused tools-admin-login-redirect-points-into-the-admin-mount and
pinned by TestToolsAdminPostureRedirectsIntoTheAdminMount, which fails the day
the core answers 401 there.
Tests: a flagged is-admin-flag user reaches track by bearer (202); by cookie
it is 403 CSRF_INVALID without the header and 202 with it, also under a
cookie prefix; the refused non-admin session now sends the header and asserts
the console's {"error":"Forbidden"} body, so the session guard or a CSRF
refusal cannot pass for it; the legacy-secret comment says the guard compares
strings. TestWireDeviationIDsArePinned and the deviations index gain the new id, and the
index test's header says each product id opens its own row.
Review findings (PR #12: wire 1, 2, 3, 4; security 1, 2; conventions 2, 6, 9b).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… first-user policy RS-17 refuses admin.accessPolicy: first-user on every driver, so the usage text no longer names it as a reader of the user directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes (33 findings), head
|
…t (D9b) Closes §8.10: the SQS consumers own idempotency. The reserved IDEM# shape is kept but the bare attribute_not_exists(PK) is not, because every retry of one queued delivery carries the same id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four states and a lease on IDEM#webhook#<deliveryId>: a claim is granted when nobody has tried, the last try failed, or the last lease lapsed; delivered and abandoned are absorbing. A lost claim is classified from the pre-image, with no read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The attempt crosses whole — URL, the signed header set, the body bytes base64-encoded — in a product-owned envelope with a schema number; the resolved retry delay and the correlation id ride as message attributes so the worker never reads the webhook store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hedule (D9b) A Lambda on the webhook queue with ReportBatchItemFailures: claim the delivery in the ledger, POST with the core's own HTTP deliverer and the bytes the core signed, then acknowledge, back off by RetryDelay x 2^attempt through ChangeMessageVisibility, or hand the message to the DLQ with a reason. Attempts are counted by the ledger, not by ApproximateReceiveCount. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er (D9b) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Url is set (D9b) WebhookSender.Deliverer becomes the SQS deliverer; webhookDefaults records each config's resolved RetryDelay for it and counts the attempts a request owes, and App.Handle waits (bounded) for those enqueues before the response reaches the runtime. Unset, D9a's in-process deliverer stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed delivery id (D9b) outgoing-webhook-delivery-races-the-response is narrowed to the default, unqueued configuration rather than retired: the queue is opt-in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… alarm (D9b) Gated on EnableWebhookQueue (with EnableTools); off adds nothing. The only new alarm is the DLQ's depth, gated on its own switch as well as EnableAlarms, so the set is nine without the queue and ten with it. deploy.sh checks every CodeUri artifact the template names; CI builds and size-checks both zips. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ver URL (D9b) The suite runs the receiver and chose the secret, so it can verify X-Webhook-Signature; the stack reaches it through a tunnel named by AWESOME_AUTH_CONTRACT_WEBHOOK_RECEIVER_URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the webhook queue (D9b) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… time (D9b) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… managed SQS policy (D9b) A function with an SQS event source and no Role gets a SAM-generated role with AWSLambdaSQSQueueExecutionRole attached, which grants receive and delete on every queue in the account. WebhookWorkerRole now holds the worker's four statements (queue, DLQ, ledger partition, own log group), each with a Sid, and no managed policy. template_test.go asserts the explicit Role, no Policies on the function, no ManagedPolicyArns, and the exact action list of every statement instead of a deny-list (review: security 1, security 8). The alarm-gate shape check now requires AlarmsEnabled as the first term of the !And (conventions 9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ue, and count attributes in the size (D9b) ErrWebhookTooLargeToQueue is permanent, so waiting on the core's back-off held the response for the full two-second flush bound; the count now settles at attempt 0 and never again for that delivery, so the core's later retries cannot release another delivery's wait. The size check adds the attributes' names, types and values, as SQS counts them. The flush warning is a constant the test matches on, and the production correlation function is named so the test runs it on a queued delivery's context. (review: security 2, wire 6, wire 7, wire 10) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a truthful reason for an abandoned duplicate (D9b) Every path that hands a record back to the queue now checks the receive ceiling, not only a refused POST: on the last receive a duplicate bounce is dead-lettered as busy-at-ceiling and a ledger outage as ledger-unavailable, so a DLQ message without a reason means the worker did not finish its last receive (review: wire 4). A failing message whose next attempt would come within twelve hours of the queue's retention is dead-lettered as expiring, because SQS deletes an expired message without redriving it; the worker learns the retention from AWESOME_AUTH_WEBHOOK_QUEUE_RETENTION_SECONDS, pinned equal to the queue's MessageRetentionPeriod (security 4). A receive that finds the ledger abandoned hands its copy over as abandoned-earlier instead of claiming exhausted (security 6). The template's comments, the alarm description, the DLQ output and the concurrency parameter now say what the DLQ keeps and for how long, what a message carries, what the alarm cannot see, and that concurrency bounds the rate and not the total (security 3, 5, 9). The "never earlier" claim is qualified for an SQS duplicate copy (wire 9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te cause, and price who fills the queue (D9b) The register, config reference §17.4 and data-model §1.9 said a receiver sees one delivery id per event and subscription; an enqueue whose client deadline fires after SQS stored it is re-enqueued by the core under a fresh id, so the id is one per queued message and both entries now say so (review: wire 1). The at-least-once entry names that duplicate, the early retry of an SQS duplicate copy (wire 9), the receive ceiling that bounces and ledger outages spend (wire 4), the expiring hand-off and the silent loss past retention (security 4), and the alarm only when EnableAlarms is on (wire 5). §17.4 lists every DeadLetterReason, what a queued message holds and for how long (security 5, 9), what a track caller can still cost (security 2) and that the flush warning is on the request's own log line (conventions 11). Cost model §3.3: concurrency bounds the rate and not the total, the monthly worker-duration ceiling, who can fill the queue without a credential, and the 64 KB billing unit (security 3). The ledger's TTL comment and §1.9 say where a backlog can outlive the window (security 7); §9 lists the ledger among the items written (conventions 8); the store file pins its own shape (conventions 2); the stale "nine alarms" counts say ten with the queue (conventions 7); and the D9b lines in tools.go are marked (conventions 12). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ch one it builds, teardown names the queues (D9b) deploy.sh now needs every CodeUri the template names, so the README's build line passes LAMBDAS="auth webhook-worker" and build-lambda.sh's comment no longer says the default serves deploy.sh (review: conventions 1). go.yml's LAMBDAS is job-level with a D9b marker, and the size check loops over it, so a function cannot be built and not checked (conventions 5). The D9b Rule no longer splits the admin-root comment from its Rule (conventions 6). teardown.sh names the webhook queues and the dead-lettered deliveries a stack delete destroys (conventions 13). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…out tools or a webhook store, and checks the wire (D9b) The case's trigger is a login, which reaches a webhook only because of the product deviation library-events-are-bridged-into-the-tools-fan-out; its Doc now says so and cites webhook-sender.ts for the delivery's wire alone, and "one POST per login" is gone: the assertion is at least one, as at-least-once delivery requires (review: wire 2). It skips with the reason on an anonymous POST <tools>/track answering 404 and on a 404 from POST <admin>/api/webhooks (wire 3, conventions 10; CapTools is not added, because it is absent under the default apiKey posture where the bridged login is delivered all the same). It asserts a v4 delivery id, the toISOString timestamp equal to the body's, the envelope's member set and the event equal to its header (wire 8). The suite README lists webhook-receiver among the REQUIRE capabilities and has the case's row (conventions 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s correlation id; log a retry only when it is one (D9b) The correlation test now also builds the webhook queue without an injection and asserts its SQS deliverer reads the login's X-Correlation-Id from the queued delivery's context (review: wire 7). The worker logs "retrying" inside handBack, on the hand-back branch alone, so a record dead-lettered at the ceiling or as expiring no longer logs both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a4da656 to
3774a0b
Compare
|
Rebased onto the current head of #12 (
This is the first of three branches rebased into a line: #13 goes on #12, #14 on #13, and #15 on #14. The shared-file reconciliations between blocks are made on #14 and #15, not here. After #12 is squash-merged, this branch needs 🤖 Generated with Claude Code |
…list, one statement of the counts D9d now sits on D9b (#13) rather than beside it, so the shared pieces each block wrote on its own are made one: - infra/sam/template_test.go: one Conditions parser (load's tpl.conditions); D9b's conditionDefinition is gone and gatedOnAlarmsEnabled reads the parsed map, requiring !And [!Condition AlarmsEnabled, !Condition <feature>] with AlarmsEnabled first. offByDefaultAlarmGates lists WebhookQueueAlarmed (D9b) beside ScriptRunnerAlarmsEnabled (D9d), so the default count stays nine and the gate test checks both switches default to off. - docs/cost-model.md §3.3: one opening paragraph states the rule and the numbers once — nine by default, one per optional function, **All-on total: 11 alarm metrics** — and every "ten with ..." / "the tenth" / "the eleventh" in the cost model, the SAM README, the template's comments, the README and the config reference now points there. - The seam header in cmd/auth/tools.go names both filled seams. - scripts/build-lambda.sh's example builds the three functions that exist (it named a cmd/sse that does not). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st model §3.1 and §3.3, SAM README, contract cases docs/sse.md states what a client receives and is promised across a reconnect. The config reference argues the SSE function, the event log's two knobs, AuthType NONE against AWS_IAM behind OAC with the cost of each, the reservation of 20, the CloudFront behaviour and the URL clients open, and CORS by the tools mount's rule. The cost model's §3.1 is now what is deployed: 128 MB, the counted per-poll read, the publishing writes, refused requests; §3.3 prices the alarm set (nine by default, ten with SSE, eleven with SSE and the webhook queue). The contract suite gains sse/stream-emits-named-frames-with-rawData and sse/resume-replays-after-last-event-id, read over plain net/http with a bounded deadline, and run against the Function URL as well when AWESOME_AUTH_CONTRACT_SSE_URL is set. Every shared-file edit is placed away from D9b's (PR #13) hunks, so the two merge mechanically; the data-model design moves into §1.5 for that reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, the counts stated once, the stale D9c lines refreshed D9c now sits on D9d (#14), and through it on D9b (#13), so: - infra/sam/conditions_test.go, the verbatim copy of D9b's helpers, is deleted. The template test has one Conditions parser, and SseAlarmsEnabled is in offByDefaultAlarmGates with EnableSse. D9c's own default count and TestFeatureGatedAlarmsAreOffByDefault were dropped during the replay for D9d's mechanism, which checks every listed gate defaults to off. - docs/cost-model.md §3.3: the one opening paragraph names all three optional alarms. **All-on total: 12 alarm metrics**, asserted. The SSE row, §1, §3.1 and §5 point there instead of saying "the tenth" or "ten with SSE". The SAM README and the template's comments do the same, as do D9b's leftover "ten with the queue" / "$1.00 with the webhook queue". - SseFunction's environment carries the auth function's inbound-webhook trio line for line. D9d changed AWESOME_AUTH_TOOLS_INBOUND_WEBHOOKS from 'false' to the EnableInboundWebhooks switch and added the runner and timeout lines, and D9c's TestTheSseFunctionIsASubsetOfTheAuthFunction holds the SSE environment to the auth function's. The SSE function never mounts the route (streamToolsOptions), and naming the runner grants it nothing, but RS-15 would refuse its cold start with the route on and no runner. - TestCIBuildsEveryFunctionTheTemplateDeploys (D9d) compares the template's CodeUri list as a set: the SSE function deploys the auth zip. - The seam header in cmd/auth/tools.go lists the three seams as filled. - The three lines left stale on purpose are refreshed: the config reference §17.8 D9c row (and its heading), the RS-14 note in internal/config/errors.go, and the tools-surface log line's `stream` attribute, which now says where the stream is served. So is the tools.sse.enabled knob-gap remedy that still said "or for D9c". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dipende dalla PR #12: unire prima quella, poi questa con Squash and merge.
Stacked on PR #12 (
4025f04); this PR's own commits are4025f04..3774a0b. The review fixes (33 findings) and the line-up on #12 are listed in the PR comments; where this description and the comments disagree, the comments are current.What this does
D9a left
WebhookSender.Delivererat the core's in-process HTTP deliverer, which on Lambda races the response freeze (outgoing-webhook-delivery-races-the-response). This block fills that one seam, opt-in:tools.outboundWebhooks.queueUrl([new],AWESOME_AUTH_TOOLS_OUTBOUND_WEBHOOKS_QUEUE_URL). Empty (the default) keeps D9a unchanged. When set, the sender's transport isawsintegration.SQSWebhookDeliverer: the core's fully built, signed, numbered attempt goes onto SQS whole (URL, full header set with the signature, body bytes base64'd) in a product-owned, schema-numbered envelope. The core's envelope, headers, signature and numbering are unchanged.webhookDefaults.FindByEventruns synchronously on the request goroutine before the emitter spawns its goroutines, so it counts the settlements the request owes.App.Handlewaits for them (at most 2 s) before handing the response to the runtime. Without this, "at-least-once" would still race the freeze.cmd/webhook-workeris a new binary and artifact. It runs a 128 MB arm64 Lambda on the queue (batch 10, records concurrent,ReportBatchItemFailures). Per record: claim in a ledger → POST with the core's ownHTTPWebhookDeliverer, bytes verbatim (plusX-Correlation-Id, as in process) → then either ack, back off withChangeMessageVisibility=RetryDelay × 2^attempt(rounded up to whole seconds, capped just under 12 h), or dead-letter with aDeadLetterReason(exhausted/receive-ceiling/malformed) andLastStatus.internal/store/dynamodb/webhook_deliveries.go) is designed first indata-model.md§1.9 and closes §8.10. The item isPK=IDEM#webhook#<deliveryId>,SK=IDEM, with a 24 h TTL. It keeps the reserved key shape, but not the bareattribute_not_exists(PK): every retry of one queued delivery carries the same id, so a boolean key would forbid every retry. Instead it has four states (claimed / failed / delivered / abandoned) and a lease equal to the invocation deadline. A duplicate that finds a live claim is not acked. A lost claim is classified fromReturnValuesOnConditionCheckFailurewith no read.EnableWebhookQueue(defaultfalse; takes effect only withEnableTools). A Rule refuses it withoutEnableTools.Retry-count decision
Remaining == Retries(), so the count travels in the body.RetryDelay()is not on the attempt. It is taken from the configuration the core itself resolved:webhookDefaults.FindByEventrecords each returned config'sRetryDelay()by ID just before the sender runs, and the deliverer carries it as theRetryDelayMsattribute. Two alternatives were rejected. The stack-wide default would silently ignore per-row values, which would be a deviation. A store read has no by-id getter in the core, needs the tenant, and could disagree withRemaining. An attempt the snapshot never saw falls back totools.outboundWebhooks.defaults.retryDelayMs.claims, notApproximateReceiveCount. A duplicate receive is not an attempt.maxReceiveCount(WebhookQueueMaxReceiveCount, default 12 = the schema's max 10 retries + 1 + 1 slack) is a ceiling/backstop. The worker enforces the per-subscription count and dead-letters explicitly. A row wanting more attempts than the ceiling allows is dead-lettered at the ceiling withreceive-ceiling. Only messages the worker crashed on are redriven silently.Alarm decision (rule 10)
Option 1: only the alarm that catches this function's failure mode.
WebhookDeadLetterAlarmfires on DLQApproximateNumberOfMessagesVisible ≥ 1(Maximum, 300 s,notBreaching). It is gated onWebhookQueueAlarmed: !And [AlarmsEnabled, WebhookQueueEnabled]. That gives nine alarms without the queue and ten with it, still inside the free ten.template_test.go's counting rule is unchanged. Its condition check is widened to accept a feature condition defined as!And [!Condition AlarmsEnabled, …], because an alarm on a conditional queue cannot be gated onAlarmsEnabledalone. The worker's errors/throttles/duration/concurrency are deliberately left unalarmed: a crash loop ends in the DLQ, and throttling only delays delivery (14-day retention).With this block alone the set is exactly 10. D9d (#14) and D9c (#15), lined up on top of this PR, change the counting rule to "alarms enabled by default" (9) and assert the all-on total (12) against cost-model §3.3; this PR keeps the rule as it was.
New AWS resources (only with
EnableWebhookQueue+EnableTools)WebhookQueue(SSE-SQS, visibility 180 s = 6 × worker timeout, 14-day retention, redrive → DLQ)WebhookDLQ(SSE-SQS, 14-day retention)WebhookWorkerFunction(128 MB arm64, 30 s, own IAM)WebhookWorkerLogGroup(/aws/lambda/<stack>-webhook-worker,LogRetentionDays)WebhookDeadLetterAlarmThe per-attempt cost is ≈ USD 4.60 per million attempts, more than half of it the ledger's 2 WCU. The full table is in
docs/cost-model.md§3.3.IAM:
EnqueueOutgoingWebhooks(sqs:SendMessageon the queue).ConsumeWebhookQueue(receive/delete/change-visibility/get-attributes on the queue),DeadLetterWebhooks(sqs:SendMessageon the DLQ),WebhookDeliveryLedger(dynamodb:UpdateItemon the table only fordynamodb:LeadingKeysIDEM#webhook#*).Deviations
queued-webhooks-are-delivered-at-least-once: a guarantee the reference does not give (duplicates are possible, there is a DLQ, and the schedule has transport limits).queued-webhook-retries-reuse-the-delivery-id: retries carry the id the core minted. The reference mints one per attempt.outgoing-webhook-delivery-races-the-response: it now applies only whilequeueUrlis unset, because the queue is opt-in.RS rules: none taken. RS-19 stays unused. A
queueUrlwith the tools block or webhook store off is reported as an unwired knob, not refused.Shared files touched (each edit contiguous, marked D9b)
cmd/auth/tools.go:toolsWiring.queue, anewToolsWiringparam, one block aftertw :=, thewebhookDefaults.queuehook, the log line, and a header paragraph.cmd/auth/app.go:Options.WebhookDeliverer, the call site,Handleflush, andunwiredKnobs.cmd/auth/tools_test.go: a D9b cases block inTestUnwiredKnobsIsExactlyTheDocumentedList.cmd/auth/deviations.go,deviations_test.go.infra/sam/template.yaml: parameter group, three params, two conditions, a Rule, the auth env var and IAM, a resources block, the alarm, and two outputs.infra/sam/template_test.go: the condition check and a D9b section..github/workflows/go.yml:LAMBDASplus a size check for both zips.scripts/deploy.sh: checks/builds everyCodeUriartifact the template names.scripts/toolchain.sh: two contract env vars.internal/config/{config,env,validate,defaults_test}.go.docs/spec/{config-schema,data-model}.md,docs/{config-reference,cost-model,deviations}.md,infra/sam/README.md,test/contract/README.md.go.mod/go.sum:service/sqs v1.52.0, no core SDK bump.Tests
exhausted+LastStatus. Timeout is tested with a log that never quotes the URL. Also covered: visibility arithmetic, malformed → DLQ, receive-ceiling, and a busy duplicate not acked.Handlereturns. The defaults snapshot is tested, and a failed last attempt does not hold the response.tools/outgoing-webhook-reaches-the-receiver-signed, runs behindAWESOME_AUTH_CONTRACT_WEBHOOK_RECEIVER_URL(+_LISTEN). The suite runs the receiver behind a tunnel and chose the secret, so it verifies the signature itself.Gate green on
./...with no skips.LAMBDAS="auth webhook-worker"builds both: auth 9.7 MB, worker 5.0 MB, both under the 12 MiB budget. No AWS command was run.Concerns not settled
RedrivePolicy.maxReceiveCount: !Ref WebhookQueueMaxReceiveCount(a Number parameter inside a JSON-typed property) is validated by reading only. It should be checked at the first deploy.abandoned. §17.4 documents deleting theIDEM#webhook#<id>item first.NumberOfEmptyReceivesafter a day.For C1 (reference 1.10.0–1.10.8, not implemented here)
src/tools/webhook-sender.tsitself did not change in 1.10.x. Related items:(req.body ?? {})across the tools router, so a body-lessPOST /track//notifyanswers 200/400 instead of 500.AuthToolsOptions.sseDistributor, plus the warning when bothsse: trueand a distributor are set (D9c's transport).correlationId1–128 chars of[A-Za-z0-9_.:-],email≤ 320,AUTH_OAUTH_CONFLICTfields limited).identity.user.email.changed({ oldEmail, newEmail }), which webhooks can subscribe to.node:vmis not a secure sandbox for inboundjsScript(D9d's).Upstream: nothing needed. The core's
WebhookDelivererseam was sufficient.🤖 Generated with Claude Code