Repository navigation
Better cli - #3
Merged
Merged
Conversation
ascandone
added a commit
that referenced
this pull request
Sep 18, 2026
…rements The doc said the generator rarely produces a save-overdraw followed by a bounded-overdraft draw on the same account. Measured over the same 3000 seeds it produces the ingredients constantly and the combination never: 686 saves, 2284 bounded overdrafts, 130 on a shared account, 7 in order, 2 that also overdraw, 0 complete. The reason is structural, so say it: statements are sampled independently, and a shape needing two of them to hit the same (account, asset) in order is quadratically suppressed.
3 tasks done
ascandone
added a commit
that referenced
this pull request
Sep 25, 2026
* test: differential testing against the legacy ledger machine Runs generated programs through both the numscript interpreter and a vendored copy of the ledger repo's `internal/machine`, and compares what moved. - `internal/oracle/` is that vendored copy, in its own module so it cannot drift into the interpreter's dependency graph. `smoke_test.go` checks the vendoring itself did not change behaviour. `DIVERGENCES.md` records every difference from ledger main and why it is there; `DIFFERENCES-BY-EXAMPLE.md` is the same list as runnable scripts. - `internal/difftest/` is the harness, also its own module: `Compare` (what counts as a real difference vs a posting-granularity one), the seeded `TestDifferentialSweep` gate, `TestKnownBugRepros`/`TestKnownOpenDivergences`, and `FuzzDiff` with its corpus. CI gets a `Difftest` job, because both are separate modules and the root `go test ./...` reaches neither. It runs the deterministic sweep, not the fuzzer: `-fuzztime` expiry surfaces as a failure with no divergence behind it. The sweep is green -- 0 divergence classes and 0 tolerated over 3000 seeds. Getting there meant changing the oracle's `kept` to consume its funding, the way the interpreter does and unlike ledger. That is a deliberate loss of independence on `kept` and is written up as DIVERGENCES.md §2 ②. * refactor(gen): trim comments and unexport the internal-only helpers Drops the running commentary about the generator's Haskell ancestry (Gen.hs/Numscript.hs/Utils.hs, QuickCheck's `frequency`, `Ratio Int`) — none of it is a fact about this code — along with comments that restate the field they sit on and rationale that had grown into prose. Kept: what cannot be read off the code — why `max`-clause amounts are never negative, why both pools are sorted before being indexed, why the var-collision shapes are biased for. `GenerateScriptAST`, `ToBuilder` and `ToBuilderScript` were exported but used only inside the package; `GenerateScript` and `RandFromBytes` are the entry points. Unexported. Also removes references to DIFFTEST_HANDOFF.md, which is not in the repo, and deletes a superseded duplicate doc comment on genPresetMetadata. internal/gen landed in #194 before this cleanup was ready, hence the separate commit. * docs(oracle): merge the two divergence docs into one DIVERGENCES.md and DIFFERENCES-BY-EXAMPLE.md covered the same four differences at two levels of detail, and the second was written on top of the first rather than replacing it, so both had to be kept in sync by hand. One file now. 472 lines down to 244. Each difference gets a runnable script, what all three engines do with it, why, and the open question. The vendoring mechanics and the drift check move to the end, where they belong. Section numbers become #1-#4, which is what the code comments now cite. * oracle: drop the OP_TAKE guard, tolerate the difference in Compare instead The oracle carried a negative-amount guard on OP_TAKE that ledger does not have. That made vm/machine.go a behaviour fork, and hid the cost: the oracle silently stopped being ledger on every script with a negative send amount. Copy ledger exactly instead, and absorb the difference in Compare under a named tolerance. Both engines reject those scripts and no money moves either way; only the error differs (`insufficient funds` vs `Cannot send negative amount`). The tolerance is one-directional: the interpreter naming a negative amount while the oracle blames the funds. The reverse is DIVERGENCES.md #4, where the oracle rejects and the interpreter silently zeroes the clause and runs short, and that must stay a mismatch. TestMissingFundsClassificationMismatchStillCaught caught the symmetric version of this and is why it is not. The sweep prints how often it fires -- 161 of 3000 scripts -- so the cost is countable on every run instead of invisible. vm/machine.go is now byte-identical to ledger apart from the vendored types, leaving `kept` as the oracle's only behavioural divergence. Also records in DIVERGENCES.md #4 why ledger PR #2079 was closed: clamping a negative `max` needs a new opcode, and adding one to the machine is too aggressive for what it buys. * docs(oracle): replace the "rarely produces" claim about #3 with measurements The doc said the generator rarely produces a save-overdraw followed by a bounded-overdraft draw on the same account. Measured over the same 3000 seeds it produces the ingredients constantly and the combination never: 686 saves, 2284 bounded overdrafts, 130 on a shared account, 7 in order, 2 that also overdraw, 0 complete. The reason is structural, so say it: statements are sampled independently, and a shape needing two of them to hit the same (account, asset) in order is quadratically suppressed. * improve generator * fix review issues * apply bot suggestions * difftest: narrow the one-sided-runtime-failure tolerance to the known negative-max gap Compare tolerated any one-sided runtime failure once both sides agreed on missing-funds classification, not just the known negative `max` clause divergence (DIVERGENCES.md #4). A real bug where one engine committed money the other refused to move, for an unrelated reason, would have passed through silently. Add SideResult.NegativeMaxReject, set by run_oracle.go when the failure is ledger's OP_TAKE_MAX guard rejecting a negative max clause (untyped upstream, so matched by its fixed error text rather than by type), and require it before tolerating the asymmetry. Any other one-sided failure is now a mismatch. Renamed the tolerance from "one-sided runtime failure" to "negative max clause" to match, and updated DIVERGENCES.md and the tests that asserted the old name. * made the err more precise
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.