feat: development-only devtools hook — adapters report which signals each event wrote - #1203

What

In development, each LangGraph and AG-UI agent reports which of its signals each event wrote. The AG-UI DevTools Chrome extension reads these reports to fill its Signals tab: a matrix with signals as rows and events as columns. The emitter lives in @threadplane/chat as the private ɵcreateDevtoolsEmitter. Both adapters already depend on that package, so this needs no new package or release change.

The contract

After handling an event, the agent dispatches this on window:

window.dispatchEvent(new CustomEvent('threadplane:devtools', { detail }));

interface ThreadplaneDevtoolsReport {
  v: 1;
  agent: string;              // random per agent instance (crypto.randomUUID, counter fallback), ≤ 64 chars
  adapter: 'langgraph' | 'ag-ui';
  seq: number;                // per agent, starts at 1, one per dispatched report
  eventType: string;          // protocol event name, or a pseudo-event label; truncated to 128 chars
  wrote: string[];            // distinct names from the adapter's vocabulary, in first-write order, 1..15
  tMs: number;                // performance.now() when the event began
}

• LangGraph vocabulary: the StreamSubjects keys without $: status values messages error interrupt interrupts branch history isThreadLoading toolProgress toolCalls messageMetadata subagents queue custom.
• AG-UI vocabulary: the ReducerStore fields plus the interrupt-session signal: messages status isLoading error toolCalls state interrupt customEvents activities interruptSession. usage and pendingClientToolCallIds are not in the contract and are not reported.
• eventType: the LangGraph StreamEvent.type or the AG-UI event.type. Writes that no protocol event caused carry a pseudo-event label instead: run:start run:end history reset submit queue branch.

The extension validates exactly this shape: own properties only, the exact key set, the vocabulary for each adapter, and the length limits. The emitter enforces the same rules, so it can never send a report the extension would reject.

Privacy model

• Names and timing only. The instrumentation wraps BehaviorSubject.next (LangGraph) and WritableSignal.set/update (AG-UI) and passes each argument through untouched. It never reads the written value or a signal's current value, and it never runs an update callback itself. Nothing from the conversation is in a report; the extension already has the wire traffic.
• Dev-only. The emitter returns null unless (typeof ngDevMode === 'undefined' || ngDevMode) && isDevMode(), which is the gate the telemetry runtime already uses. It also returns null outside a browser. When the emitter is null, nothing is wrapped and the adapters behave exactly as before. A production build defines ngDevMode = false, which removes the whole hook from the bundle (see the bundle check below).
• Fire-and-forget. A CustomEvent gets no reply, so the page cannot tell whether anything is listening, and the extension's presence stays hidden. An event that wrote nothing produces no report. If dispatch throws, the error is swallowed.
• Opt-out: setting window.__THREADPLANE_DEVTOOLS_DISABLED__ = true (only the value true counts) turns the hook off. It is checked when the agent is created, so no wrappers are installed, and again before each dispatch, so setting it later also works.

How writes are attributed

• Per event. LangGraph brackets processEvent and AG-UI brackets onEvent. Every write made while the bracket is open belongs to that event, including the adapter's own follow-up work: subagent settling, interrupt-session publishing, rollback.
• Outside an event. Run start and settle, history refresh, thread reset/switch, queue changes, setBranch, and retry/regenerate/submit preambles are bracketed under the pseudo-event labels.
• Nesting. Brackets nest by joining: a helper that brackets itself and is called during an event becomes part of that event's report.
• Known gap. A write outside every bracket is not reported. Today that is only AG-UI client-tool settlement (clientTools.resolve/settle writing toolCalls).

Tests

• libs/chat/src/lib/devtools/devtools-emitter.spec.ts (14 tests):
  • the report shape against a verbatim copy of the contract
  • dedupe and order, seq per agent, no report for zero names, nested brackets, the vocabulary filter, the 128-character limit
  • a bracket that throws still closes; a dispatch failure is swallowed
  • gating: isDevMode() false, ngDevMode = false, opt-out before creation and later, only true counts
• libs/langgraph/src/lib/devtools.spec.ts (8 tests, through agent() with MockAgentTransport). Representative wrote lists:
  • messages tuple → ['messages', 'messageMetadata', 'subagents', 'toolCalls']
  • root values with messages → ['values', 'messages', 'subagents', 'toolCalls']
  • updates → ['values']; custom → ['custom']; error → ['subagents', 'error', 'status']
  • a full run: run:start ['status', 'error', 'custom', 'toolProgress', 'messages'] → values ['values'] → run:end ['subagents'] → run:end ['status']
  • setBranch → branch ['branch']; switchThread → one reset report; retry → submit ['error'], then run:start; history refresh → history reports
  • two agents get separate ids and seq sequences, and every report stays inside the contract
  • nothing is dispatched outside dev mode or after the opt-out
• libs/ag-ui/src/lib/devtools.spec.ts (7 tests, through toAgent() with a scripted FakeAgent). Representative wrote lists:
  • RUN_STARTED → ['status', 'isLoading', 'error', 'interrupt', 'customEvents', 'activities']
  • TEXT_MESSAGE_CONTENT → ['messages']
  • STATE_SNAPSHOT and STATE_DELTA → ['state', 'messages']
  • TOOL_CALL_START → ['toolCalls', 'messages']; TOOL_CALL_ARGS → ['toolCalls']; CUSTOM → ['customEvents']
  • RUN_FINISHED → ['messages', 'status', 'isLoading', 'interruptSession', 'interrupt']
  • RUN_ERROR → ['messages', 'status', 'isLoading', 'error', 'state']
  • stop() → run:end ['state', 'messages', 'status', 'isLoading', 'error']; regenerate() → submit ['messages']
  • the same contract, dev-mode and opt-out checks as LangGraph
• No value is ever read. Both adapters have a test that writes a value that cannot be touched (a Proxy whose every trap throws). The LangGraph test writes it through a subject whose value and getValue() throw. The AG-UI test writes it through a signal whose getter throws, and calls update with a callback that throws if it is run. In both, the write must succeed, the value must reach the subject or signal as the same object, and wrote must receive only the name.
• Existing suites. The existing chat, langgraph and ag-ui suites run with the hook enabled (tests run in dev mode) and all pass unchanged: chat 1213, langgraph 489, ag-ui 554.

Bundle check

libs/chat/scripts/verify-devtools-bundle.mjs (run with nx run chat:test-devtools-bundle; added to the Library CI job after the production build):

• Bundles the built @threadplane/chat, @threadplane/langgraph and @threadplane/ag-ui with esbuild (minified, other imports external), with both adapters' provideAgent as the entry.
• Production (ngDevMode=false): asserts the bundle contains neither threadplane:devtools nor __THREADPLANE_DEVTOOLS_DISABLED__.
• Positive control: the same bundle without the define must contain both.
• Mutation check: I removed the ngDevMode gate from the built chat bundle and ran the check; it failed with "the production bundle still contains threadplane:devtools".

Verification run locally

• nx run-many -t lint,test,type-tests --projects=chat,langgraph,ag-ui
• nx run-many -t build --projects=chat,langgraph,ag-ui --configuration=production
• nx run chat:test-devtools-bundle
• nx run langgraph:runtime-quality, langgraph:runtime-type-tests, ag-ui:runtime-quality, ag-ui:runtime-type-tests
• scripts/react-parity/inventory.mjs --check (baseline refreshed; dispositions added for the new private seam), verify-boundaries.mjs, and the react-parity script specs
• check-dx-coverage.mjs
• generate-api-docs: no diff, because the new exports are @internal

🤖 Generated with Claude Code