trace-interop

Reproducible evidence for a proposed Ethereum trace_* specification, with per-client change reports.

Each bar is one client's development build across every decision in the ledger, regenerated with each capture. See progress by client for the counts, trend and definitions.

Several Ethereum clients expose the Parity trace_* methods, but the same request can mean different things or return different shapes. Tracing tools carry client-specific workarounds, and maintainers have no shared contract to test against.

The execution-apis standardization discussion (#890) needs a concrete proposal and a clear view of its impact on each client. This repository supplies the draft specification, runnable cases on every client, and reports that explain what would change, why, and where to look in the code. Observed behavior stays separate from recommended changes, and the draft remains a proposal for review.

Join the trace working group on Telegram to discuss the proposal and coordinate client work.

Client maintainers

1. Your client's report: each difference from the draft, with a minimal example, the proposed change and links into your source. 🛠️ marks differences already covered by a submitted fix.
2. Decision pages: the recommendation, rationale and cross-client observations behind each proposed change.
3. Client fixes: each related PR, the decisions it addresses and how far it has reached toward a verified measured build.
4. Changes since the previous matrix: verdicts that changed for each client build since the last capture.

Specification reviewers

• Draft specification PR #895 and its source branch, which this repository pins rather than duplicating.
• Decision ledger: every open question, its policy status and per-client verdicts.
• Source review: the draft compared with Parity, current client source and the sibling eth_* methods; its recommendations are adopted in the ledger and draft.
• Upstream acceptance: what execution-apis requires to accept the proposal, and lessons from recent reviews.
• Geth draft implementation: an experimental fork implementing the draft, evaluated separately from the native clients.

Contributors

• Reproduce and review: builds, frozen chains, captures, report generation and evidence storage.
• Reviewing a proposed rule: how to separate API decisions from execution defects and record a disagreement.

1. Fixtures. Frozen generated chains (fixtures/chains) and request corpora (fixtures/corpora) define each case, its setup controls and its independent expectations.
2. Capture. Hive runs the corpora against the latest release and development builds of Besu, Erigon, Nethermind and Reth, plus the Geth draft fork, after a freshness preflight. Foundry's Anvil, which has no Engine API, is captured on the same chains by replaying them block by block. Stateful scenarios cover reorg restoration and pruned history.
3. Evidence. Each run keeps its requests, raw responses, client logs and build lock under evidence/, checksummed and never rewritten.
4. Assessment. Responses are checked against the pinned draft schemas and against independent models of execution, gas, fees and state. Assertion models explain what each status means and what remains unassessed. Consistency laws compare each client with itself across methods that project one execution, and decision tables enumerate the draft's clauses to find cases it leaves undecided or decides twice.
5. Reports. scripts/build_reports.py regenerates the client reports, decision pages and verdict matrix from the evidence selected by reports.lock.json.

uv sync --locked
uv run trace-interop verify
uv run python scripts/check_schema.py
uv run python scripts/build_reports.py

scripts/check.sh runs the whole CI check; see usage for optional prek hooks.

On Linux with Docker, capture a current comparison of the latest stable releases, development images and Geth draft head:

uv run python scripts/run_matrix.py --output runs/current-matrix

All nine traditional methods are covered: trace_call, trace_callMany, trace_rawTransaction, trace_replayTransaction, trace_replayBlockTransactions, trace_block, trace_transaction, trace_get and trace_filter, with the call-trace, stateDiff and vmTrace output families.

The Geth fork is a candidate implementation of the draft, reported as go-ethereum_trace. It is not upstream Geth support or an independent vote for the proposal.

This is not a public testnet, performance benchmark, general EVM fuzzer or client ranking. Tests use disposable generated chains, and no mainnet synchronization is required.

Methodology: reproduce and review · stateful scenarios · assertion models · consistency laws and decision tables · reviewing a proposed rule

Decision studies: H13 signed transaction validation · mined transaction probes · H15 unsigned simulation fees · Nethermind truncated validation responses

Reviews and audits: source review · divergent decisions review · historical records: harness assertion audit · H17 assessment audit · measurement and draft review corrections

Upstream work: client fixes · upstream acceptance · Geth draft implementation