Conversation
added 5 commits
October 4, 2026 15:17
Add scripted HTTP mocks (test_mock.rs/search_mock.rs) and query/retention tests for the ClickHouse and Aglake backends, unit tests for the storage dialect/convert helpers, CLI smoke coverage for heron, and a whole-router smoke test for the h-api read paths. Lifts the crates that had no live-server coverage.
The fixtures these suites referenced were legacy gitignored captures that do not exist, so they skipped silently (and pipeline_e2e hung once fixtures were present because pair_sweeper is an infinite task). Resolve corpus-first with an LFS-aware check, drain all finite stages before aborting the sweeper, repoint fixtures to the committed corpus, retune assertions to single-turn ground truth, and #[ignore] the four tests that still need uncommitted captures.
Annotate the tests that verify Tier 0/1 scenarios so check_scenarios.py can match each declaration to an implementation.
Add a six-dimension verification policy (code/change/behavior/risk/effectiveness/system) with stdlib-only checkers: coverage-policy.json tiers, expiring waivers, scenarios.json, and a measured baseline ratchet. Wire the cheap static gates into ci.yml and the heavy measurement + changed-code gate into a separate coverage.yml workflow. Add a PR template encoding the scenario convention.
…heck cargo-mutants matches --in-diff paths against the server/ workspace root, so repo-relative paths silently select zero mutants; generate a workspace-relative patch instead. Run mutation on the PR's changed Tier 0 lines (advisory, continue-on-error) plus a weekly full run. Record the storage equivalent mutants and the single-threaded --in-place requirement.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the single "line coverage ≥ 80%" target with a risk-driven, six-dimension verification policy — and the gates to enforce it.
Policy (
docs/design/11-verification.md,verification/)Code / change / behavior / risk / effectiveness / system coverage. Tiered
coverage-policy.json, expiringwaivers.json, ascenarios.jsonbehavior matrix, and a measuredcoverage-baseline.jsonratchet.Checkers (
scripts/ci/, stdlib-only so the runner needs no PyYAML)check_coverage_policy.py— tier classification, waiver expiry/owner, floors, no-decrease,--enforce-tiers,--write-baselinecheck_scenarios.py—@scenario <ID> <kind>tags (unit/integration/e2e/fault)diff_coverage.py— changed-code coverage from lcov + JSON ×git diffcoverage.sh,mutation.sh, self-tests underscripts/ci/tests/CI
Cheap static gates in
ci.yml; the heavy instrumented build + changed-code gate + advisory mutation run in a separatecoverage.yml(so instrumentation never holds the requiredcicheck hostage); a weeklymutation.ymlruns the Tier 0 full pass.Commits
test:raise coverage on storage backends, CLI, and API smoke pathstest:fix silently-skippingpipeline_e2eand turn integration suitestest(verification):tag critical scenarios with@scenariofeat(verification):policy, checkers, and CI gatesci(mutation):calibratecargo-mutants --in-diff, arm the advisory PR checkNotable fixes
pipeline_e2eand the turn integration suite referenced legacy gitignored fixtures that do not exist, so they skipped silently — andpipeline_e2ehung once fixtures were present because the pair sweeper is an intentional forever-task. Fixtures are now resolved corpus-first with a git-LFS-aware check, the sweeper is aborted after the finite stages drain, and the four genuinely-unreproducible tests are#[ignore]d instead of silently skipping.cargo-mutants --in-diffmatches paths against theserver/workspace root, so repo-relativeserver/...paths silently select zero mutants. The patch is now generated workspace-relative, and mutation runs single-threaded--in-place(required becauseh-commonreads the repo-rootVERSIONviainclude_str!).Test plan
cargo test --workspacecargo check --workspace --all-targetspython3 scripts/ci/check_scenarios.py→18 defined, 20 tag(s), OKpython3 scripts/ci/check_coverage_policy.py --staticrust/consolejobs as required checks once merged--enforce-tiersstays on (currently waived for the crates still ratcheting)Follow-ups
coverage-baseline.jsonon merge tomain(manual--write-baselinefor now).