Repository navigation
feat(gen): seed-driven numscript program generator - #194
Merged
Merged
Conversation
Extends the script builder so it can express the shapes the differential generator needs to emit: - nested origins (`origin.go`), rendered in dependency order and declared as vars when interpolated - allotment sources and destinations (`allotment.go`) - `send*` / uncapped-amount statements and the remaining statement forms (`statement_extra.go`) The expression layer moves from a bare string to a typed representation so a monetary, an account and a number are no longer interchangeable at the call site.
`internal/gen` turns a seed into a numscript program plus the inputs it needs (vars and starting balances), for property and differential testing. Programs are built as an AST and rendered through `builder`. Generation is a pure function of the seed: the same seed always yields the same program, vars and balances, so a failure reported as a seed number is reproducible from the number alone (`determinism_test.go`). Every value type can appear as an inline literal or as a var, metadata is read back as each of its types, and `cleanup.go` drops the shapes that are not valid numscript rather than emitting them and filtering later.
This was referenced Sep 17, 2026
Azorlogh
previously approved these changes
Sep 18, 2026
ascandone
enabled auto-merge (squash)
September 18, 2026 09:43
Azorlogh
approved these changes
Sep 18, 2026
3 tasks done
ascandone
added a commit
that referenced
this pull request
Sep 18, 2026
Drops the running commentary about the generator's Haskell ancestry (Gen.hs/Numscript.hs/Utils.hs, QuickCheck's `frequency`, `Ratio Int`) — none of it is a fact about this code — along with comments that restate the field they sit on and rationale that had grown into prose. Kept: what cannot be read off the code — why `max`-clause amounts are never negative, why both pools are sorted before being indexed, why the var-collision shapes are biased for. `GenerateScriptAST`, `ToBuilder` and `ToBuilderScript` were exported but used only inside the package; `GenerateScript` and `RandFromBytes` are the entry points. Unexported. Also removes references to DIFFTEST_HANDOFF.md, which is not in the repo, and deletes a superseded duplicate doc comment on genPresetMetadata. internal/gen landed in #194 before this cleanup was ready, hence the separate commit.
ascandone
added a commit
that referenced
this pull request
Sep 25, 2026
* test: differential testing against the legacy ledger machine Runs generated programs through both the numscript interpreter and a vendored copy of the ledger repo's `internal/machine`, and compares what moved. - `internal/oracle/` is that vendored copy, in its own module so it cannot drift into the interpreter's dependency graph. `smoke_test.go` checks the vendoring itself did not change behaviour. `DIVERGENCES.md` records every difference from ledger main and why it is there; `DIFFERENCES-BY-EXAMPLE.md` is the same list as runnable scripts. - `internal/difftest/` is the harness, also its own module: `Compare` (what counts as a real difference vs a posting-granularity one), the seeded `TestDifferentialSweep` gate, `TestKnownBugRepros`/`TestKnownOpenDivergences`, and `FuzzDiff` with its corpus. CI gets a `Difftest` job, because both are separate modules and the root `go test ./...` reaches neither. It runs the deterministic sweep, not the fuzzer: `-fuzztime` expiry surfaces as a failure with no divergence behind it. The sweep is green -- 0 divergence classes and 0 tolerated over 3000 seeds. Getting there meant changing the oracle's `kept` to consume its funding, the way the interpreter does and unlike ledger. That is a deliberate loss of independence on `kept` and is written up as DIVERGENCES.md §2 ②. * refactor(gen): trim comments and unexport the internal-only helpers Drops the running commentary about the generator's Haskell ancestry (Gen.hs/Numscript.hs/Utils.hs, QuickCheck's `frequency`, `Ratio Int`) — none of it is a fact about this code — along with comments that restate the field they sit on and rationale that had grown into prose. Kept: what cannot be read off the code — why `max`-clause amounts are never negative, why both pools are sorted before being indexed, why the var-collision shapes are biased for. `GenerateScriptAST`, `ToBuilder` and `ToBuilderScript` were exported but used only inside the package; `GenerateScript` and `RandFromBytes` are the entry points. Unexported. Also removes references to DIFFTEST_HANDOFF.md, which is not in the repo, and deletes a superseded duplicate doc comment on genPresetMetadata. internal/gen landed in #194 before this cleanup was ready, hence the separate commit. * docs(oracle): merge the two divergence docs into one DIVERGENCES.md and DIFFERENCES-BY-EXAMPLE.md covered the same four differences at two levels of detail, and the second was written on top of the first rather than replacing it, so both had to be kept in sync by hand. One file now. 472 lines down to 244. Each difference gets a runnable script, what all three engines do with it, why, and the open question. The vendoring mechanics and the drift check move to the end, where they belong. Section numbers become #1-#4, which is what the code comments now cite. * oracle: drop the OP_TAKE guard, tolerate the difference in Compare instead The oracle carried a negative-amount guard on OP_TAKE that ledger does not have. That made vm/machine.go a behaviour fork, and hid the cost: the oracle silently stopped being ledger on every script with a negative send amount. Copy ledger exactly instead, and absorb the difference in Compare under a named tolerance. Both engines reject those scripts and no money moves either way; only the error differs (`insufficient funds` vs `Cannot send negative amount`). The tolerance is one-directional: the interpreter naming a negative amount while the oracle blames the funds. The reverse is DIVERGENCES.md #4, where the oracle rejects and the interpreter silently zeroes the clause and runs short, and that must stay a mismatch. TestMissingFundsClassificationMismatchStillCaught caught the symmetric version of this and is why it is not. The sweep prints how often it fires -- 161 of 3000 scripts -- so the cost is countable on every run instead of invisible. vm/machine.go is now byte-identical to ledger apart from the vendored types, leaving `kept` as the oracle's only behavioural divergence. Also records in DIVERGENCES.md #4 why ledger PR #2079 was closed: clamping a negative `max` needs a new opcode, and adding one to the machine is too aggressive for what it buys. * docs(oracle): replace the "rarely produces" claim about #3 with measurements The doc said the generator rarely produces a save-overdraw followed by a bounded-overdraft draw on the same account. Measured over the same 3000 seeds it produces the ingredients constantly and the combination never: 686 saves, 2284 bounded overdrafts, 130 on a shared account, 7 in order, 2 that also overdraw, 0 complete. The reason is structural, so say it: statements are sampled independently, and a shape needing two of them to hit the same (account, asset) in order is quadratically suppressed. * improve generator * fix review issues * apply bot suggestions * difftest: narrow the one-sided-runtime-failure tolerance to the known negative-max gap Compare tolerated any one-sided runtime failure once both sides agreed on missing-funds classification, not just the known negative `max` clause divergence (DIVERGENCES.md #4). A real bug where one engine committed money the other refused to move, for an unrelated reason, would have passed through silently. Add SideResult.NegativeMaxReject, set by run_oracle.go when the failure is ledger's OP_TAKE_MAX guard rejecting a negative max clause (untyped upstream, so matched by its fixed error text rather than by type), and require it before tolerating the asymmetry. Any other one-sided failure is now a mismatch. Renamed the tolerance from "one-sided runtime failure" to "negative max clause" to match, and updated DIVERGENCES.md and the tests that asserted the old name. * made the err more precise
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
internal/genturns a seed into a numscript program plus the inputs it needs(vars and starting balances). Programs are built as an AST and rendered through
builder.the same program, vars and balances, so a failure reported as a seed number
is reproducible from that number alone (
determinism_test.go).read back as each of its types — not only
number.cleanup.godrops shapes that are not valid numscript at construction time,rather than emitting them and filtering downstream.
Why
Second of three PRs replacing #191. Stacked on #193 (
builder/), which it usesto render. On its own it is a generator with no consumer; #195 adds the
harness that runs what it produces through two engines.
Base is
feat/numscript-builder— retargets tomainautomatically when#193 merges.
Risk
LOW — new package under
internal/, no callers outside tests.Validation
go build ./...,go vet ./..., gofmt cleangen_test.go,determinism_test.go,value_forms_test.go,meta_types_test.gogo test -count=1 ./...greenArchitecture / behavior impact
N/A.
Review focus
Determinism is the property everything downstream relies on — if a seed stops
being reproducible, every sweep failure becomes un-actionable.
determinism_test.gois where to push.
Second: whether
cleanup.gorejects shapes that are actually valid. Anything itdrops is coverage the differential sweep never gets.
Known concerns
The generator's reach is a known limit rather than a defect: it does not produce
asset scaling, account interpolation or colors, so a green sweep says nothing
about those. Recorded where it matters in #195.