Sitelet https://github.com/postcss/postcss-selector-parser/pull/363
Skip to content

test: add property tests over generated selectors and values - #363

Open
theRizwan wants to merge 1 commit into
postcss:mainfrom
theRizwan:test/property-harness
Open

theRizwan wants to merge 1 commit into
postcss:mainfrom
theRizwan:test/property-harness

Conversation

@theRizwan

Copy link
Copy Markdown
Contributor

Adds the property tests discussed in #359.

npm run test:property builds and runs them. They live in src/__tests__/property/, so npm test doesn't pick them up. PROPERTY_RUNS sets the number of generated inputs (default 2000, which takes about a second) and PROPERTY_SEED sets the seed (default 1), so any failure can be rerun exactly.

There are three generators in generate.mjs. One builds valid selectors from a small grammar (namespaces, attributes with every operator and both quote styles, nested :not()/:is(), comments, escapes, and uneven whitespace). One concatenates fragments with no grammar, for inputs the parser should reject. The third makes arbitrary strings for the setters and constructors, minus NUL and lone surrogates, since CSS can't spell those.

The properties:

  • invalid input throws the parser's own error, never a TypeError or similar
  • lossless output is the input
  • lossy output doesn't change when parsed and printed again
  • input.startsWith(node.toString().trim(), node.sourceIndex) for every node
  • a value set through the API reads back the same after printing and parsing

known.mjs lists the failures that are already understood, each with what tracks it. A known failure doesn't fail the run, but every run prints how many failures each entry explained, and says when an entry no longer matches anything so it can be deleted. At the default settings on main:

ℹ 37 known (not filed yet): an unclosed `(` at the start or after a pseudo throws a TypeError
ℹ 14 known (not filed yet): lossy output drops the whitespace before a `|`, so `a |b` becomes `a|b`
ℹ 1062 known (#359): a namespaced tag or universal starts at its name instead of its namespace
ℹ 560 known (#357): #189: a comment before a combinator is kept in the combinator's raws
ℹ 8700 known (#357): in 7.x the constructors and the id value setter write the value as given (see API.md); escaping there is parked for the next major (#360, #361)
ℹ 7 known (not filed yet): an attribute namespace of `&` prints as `[&|x]`, which does not parse

With #359 merged on top, its entry is reported as no longer seen. #358 only shows up through the lossy whitespace bug, since :hover|a isn't valid CSS and the grammar never writes it. With #358 on top, that entry drops from 14 to 11 at the default settings, and from about 135 to about 100 at 20,000 runs.

Three of the entries are new bugs the harness found, and I haven't filed them yet:

  • a |b (a b with no namespace, inside an a) prints as a|b in lossy mode, which is a b in namespace a. a > |b prints as a>|b. These are valid selectors and the output means something else.
  • (, :hover( and a:not( throw a TypeError instead of the parser's error. This goes back to at least 7.1.4.
  • an attribute's namespace set to & prints [&|x], which doesn't parse.

Happy to open issues or PRs for them, one each, whichever you prefer.

No CI job yet, as you said.

Seeded generators build valid selectors from a small grammar, arbitrary
fragment soup, and arbitrary strings for the setters and constructors.
Five properties are checked against them:

- invalid input throws the parser's own error, not a TypeError
- lossless output reproduces the input
- lossy output is stable when parsed and printed again
- `sourceIndex` points at the node's own text in the input
- a value set through the API reads back the same after printing and
  parsing

They run with `npm run test:property`, outside `npm test`.
PROPERTY_RUNS and PROPERTY_SEED set the size and seed. Failures that are
already understood are listed in `known.mjs` with what tracks them. Each
run reports how many failures every entry explains, and an entry that no
longer matches anything is reported so it can go.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant