Keep the list's implied item out of a forwarded foreign root - #495

Item 1 of #492: an implied <li> lands inside a foreign root and flips a child's namespace.

What broke

With a policy allowing ul, li, svg and textarea, <ul><svg><textarea>x</textarea></svg></ul> came out as <ul><svg><li><textarea>x</textarea></li></svg></ul>. The browser tree of that output is ul > svg > li > textarea with an HTML textarea: li is a breakout name, so the browser pops the svg at the item and everything after it is HTML. The input's tree is ul > svg > svg:textarea. Same for <a>, for <mi> and <mtext> under math, for bare text in the root (<ul><svg>x</svg></ul> gave <ul><svg><li>x</li></svg></ul>), and for an SVG tr/td. An SVG a inside an HTML link closed the link (<a><svg><a>x</a></svg></a> gave <a><svg></svg></a><a>x</a>). Every one of these was correct under a policy that drops li, because the item was then dropped and its text landed in the right place.

Why

The containment metadata implies a list item for a list's content. A forwarded SVG or MathML root has no entry on the balancer's stack, so the text or foreign child inside it was judged against the ul below and the item was opened, and emitted, inside the root. A start tag at an HTML integration point already skipped that containment; a start tag that is a foreign element did not, and text never did.

The issue's caveat held too: with the item gone, <ul><svg><tbody>D lost D, because the policy's text gate judged the foreign tbody by its HTML name, whose default is "no text" since a browser foster-parents text out of a table part. <svg><tbody>D dropped D under every policy on main; only the item had kept it under a list.

What changed

• TagBalancingHtmlStreamEventReceiver: a start tag that the output parser inserts as an SVG or MathML element skips HTML containment, exactly as one at an integration point does (the form clause there was a special case of this), as long as the input parser still names a foreign root: the output decides, so an HTML-named element at an integration point the policy dropped, or inside a forwarded table part, is foreign too and gets no list item. An unknown input context keeps HTML containment (a form under a root still closes the form before it), and so does a foreign node this receiver's lexer reads as raw text, an SVG textArea a policy writes in lower case, where elements nested inside would be read back as text. For text, and for an element with an HTML entry open above the root (HTML inside an integration point), the root is a boundary: the entries below it imply nothing, the content is judged as in a fresh body, so an option or li still gets the select or ul it is never emitted without, and the container-closing loop stops there. The boundary is the innermost forwarded root that the output parser entered and the input parser still has open, asked through a new ForeignContentContext.isNodeOpen(serial) query, so a breakout out of an inner root moves it to the next root out and one out of the last root removes it; when the input parser has none open but the output still does, because the policy dropped the breakout, the innermost emitted root stands. The link scan that ends an open link for a new one looks no further down than the boundary, since a link inside an integration point is out of the outer link's scope for the adoption agency algorithm. Formatting still resumes, as a browser reconstructs it at an integration point. A table part whose output would be foreign is forwarded as the foreign element the output parser makes of it while the input parser still has a foreign root open, which it does at an integration point the policy dropped and under HTML inside one, as a part under foreign rules already was, instead of implying an HTML table that would break out of the root in the output and turn every sibling after it into HTML; once the input parser's context is unknown the part takes the table path as before, and so does one inside a recased SVG textArea, which this lexer reads as raw text on the next pass. The forwarded part's stray end tag, which the input parser ignores, is told to the input tracker as such instead of making it give up its context, so the foreign siblings after it stay foreign; when a foreign node of that name is open further out, which the foreign end-tag algorithm pops through the integration point, the end tag is processed as usual. An option or optgroup that is itself an SVG or MathML element gets no select, since it is not an HTML option; one at an integration point still does. Breakouts are otherwise untouched: the browser pops the root for them and they are the list's content again.
• ElementAndAttributePolicyBasedSanitizerPolicy: an element the output parser inserts as SVG or MathML holds text unless the author disallowed text in that name, asking the retained template's context when one is open, as every other output namespace decision there does. Only the table-part names change hands in practice (table, thead, tbody, tfoot, tr, colgroup); a literal-content name such as style or script keeps its bar, and the prepackaged policies allow no foreign root, so nothing in Sanitizers widens.
• Tests in HtmlSanitizerTest: the reproductions and neighbouring shapes, each a fixed point whose output parses to the browser tree of the input with namespaces; hostile payloads; the gate positive and negative (disallowTextIn, style, an HTML tbody); a dropped tbody, a dropped root, and the change listener, which no longer sees a discarded li the input never had. From the reviews: links around breakouts and inside integration points, a breakout out of an inner root only and one directly in a nested root, table parts under a dropped integration point and under HTML inside one with foreign siblings after them (a link, a textarea holding markup, a hostile payload), foreign options and an integration-point option, and the exact set of names whose text the gate now keeps as foreign elements, enumerated from the containment metadata: colgroup, tbody, tfoot, thead and tr, under svg and under math. Where an output differs from its input, the tests compare browser trees with namespaces, or assert that the elements in question are SVG or MathML elements, and never HTML ones, in the output's browser tree; an unchanged output needs no tree comparison.
• change_log.md bullet.

Before and after

Policy: ul, ol, li, svg, math, textarea, a, mi, tbody, b, p, foreignObject, desc, path, div.

Every branch output parses to the input's tree, namespaces included. Under a policy that drops li, outputs are byte-identical to main except the foreign tbody text. Under Sanitizers.BLOCKS, which allows no svg, outputs are unchanged.

Verification

• ./mvnw -o -ntp -B clean verify on the final commit ec40ba1: build success, 683 tests, 0 failures (679 existing, unchanged, plus 4 new).
• Nesting limit: <ul><svg>x</svg></ul> and <ul><svg><a>x</a></svg></ul> nested 253 and 254 deep are fixed points that parse to the input's tree; one deeper the root does not fit and the text flattens, as before.
• Differential probe (probes-review3/ReviewProbe, 15,430 inputs x 44 policies, san and events modes, main e740164 against the final commit ec40ba1):
  • san: 678,920 rows compared. 6,782 differ in output. 357 rows lose every flag (the foreign tbody, tr and colgroup text kept, and first-pass structure), 60 keep fewer flags, and 56 rows on 9 distinct inputs gain one.
  • text-changed, 34 rows on 3 inputs: <style> written inside an svg inside a select, which now stays inside the svg, where the CVE-2021-42575 rule that a style inside a select holds no text closes it at once, so its raw text comes out as escaped text beside it; main moved the style out of the select and kept the raw text as a stylesheet. The third is <Svg><button><style> under a policy allowing text in style, where the stylesheet stays in the SVG style element instead of an HTML one.
  • text-lost, 3 rows on 2 inputs: the recase policies on an input whose foreignObject they drop, where a style in an option at the dropped point is now the SVG style element the output parser makes of it, holding the stylesheet as its text, escaped as the renderer escapes everything inside svg; main emptied an HTML style inside an implied select under the CVE-2021-42575 rule and wrote the text beside it, and the probe counts stylesheet text as lost. And table renamed to div on an input where a list written inside a root inside a select now stays inside the root, where the input has it, and there main's handling of that rename under a list in a select drops the cell's text, as it does for <select><svg><ul><span><tr><td>tail on main; main had moved the list out of the root.
  • Not a fixed point on the first pass, 19 rows on 4 inputs: (a) a policy that renames xmp to pre, a breakout name, inside a foreign root (11 rows, one input): the first pass keeps main's structure and the second sees the breakout; (b) the recase policies on an input whose foreignObject under a nested root they drop (4 rows): the input's strong after it is HTML at the integration point, the output's breaks out of both roots into the list, and the second pass gives it the list's item, as a breakout in a root gets; (c) table renamed to div (2 rows): an optgroup written while a select the input still has open is closed in the output gets no select until the second pass, which is the wrapper judgment Give a free option or list item its wrapper under every container, and push the select out of a kept table #497 (item 9) reworks; (d) the two policies that drop table (2 rows): text after an integration point inside a forwarded foreign row, which main dropped altogether, is kept inside the row on the first pass and after it on the second, where the inner table's dropped parts are gone.
  • events: 36,063 rows, the receiver's own event column identical on every row, no row gains a flag.
  • Earlier runs, before the reviews' changes, had 1,542, then 167, then 108 rows gaining a flag; the reviews' findings and the narrowing of the forwarding to a known input root account for the difference.

Review

/code-review on the first commit found one regression and three gaps in the boundary: after a breakout element with its own stack entry the entries above the root are HTML flow again, so a link below the root was no longer ended by a new link (<a><svg><div>x<a>y came back nested); the link scan reached below the boundary and closed and reopened a formatting element at an integration point, leaving an empty <b></b>; a table part at an integration point the policy dropped implied an HTML table inside the output's foreign root; and content below the boundary implied nothing at all, which also dropped the select an option must never be emitted without. All four are fixed above, each with a test. The review's count of the gate's widening (adding html, frame, frameset, bgsound) did not hold: those come out empty or beside the text under a foreign root on main too, and the enumeration test pins the five table-part names. Its suggestion to mark forwarded roots once instead of re-deriving them is left for the follow-up that reworks the passthrough helpers together.

A second /code-review on the reviewed commit found the boundary wrong for a breakout out of an inner root only, which should land at the integration point inside the outer root rather than below it; an HTML table implied for a part under an HTML element inside a dropped integration point, which the first fix had covered only directly under the point; the forwarded part's stray end tag making the input tracker give up its context, so the next foreign sibling got the list's item back; and a select implied for an SVG option. All four are fixed above, each with a test. Its suggestion to trust the output tracker alone once the input tracker is unknown was tried and reverted: it broke the existing test that keeps foreign forms after an unknown context out of the form element pointer, and the probe showed it forwarding parts under stray end tags where main's table path was right, so the forwarding is conditioned on the input naming a root instead.

A third /code-review found the item's symptom surviving for an HTML-named element at an integration point the policy dropped, and inside a forwarded part (<ul><svg><foreignObject><textarea>x</textarea></foreignObject><a>y</a></svg></ul> still got two list items; the option variant was no longer a fixed point): the containment skip required the input to be foreign too. It now follows the output, under a known input root, with tests for the three shapes. The review also found the forwarded part's end-tag branch ignoring an end tag that the foreign end-tag algorithm carries through the integration point to an outer foreign node of the same name, now guarded as the sibling branch is and tested; the text gate asking the outer output tracker inside retained template contents, now the template's; the browser-tree assertions in the new tests parsing the expected string twice and ignoring namespaces, now replaced; a dead overload, an orphaned javadoc and a rule inlined twice, all tidied. Its performance note, that the boundary is recomputed per event, is the follow-up already mentioned.

Left out

• A breakout inside the root (<ul><svg><b>x</b></svg></ul>) still gets the item, lexically inside the root: <ul><svg><li><b>x</b></li></svg></ul>. A browser pops the root at the li as it does at the b, and the result is the list normalisation the sanitizer applies to <ul><b>x</b></ul>. Unchanged here.
• A breakout arriving while a foreign element with a stack entry is the container (<ul><svg><a>x<li>y</li></a></svg></ul>) is judged against that entry, so an extra <ul> is implied inside the SVG a; the browser pops the a with the root and puts the item in the outer list. Main produced the same tree by another route. Popping foreign stack entries on a breakout is its own change.
• <ul><svg><foreignObject><tr><td>x, with the integration point kept, still closes the list and the root and emits the table after them, as on main: table parts keep the table path.
• Under a policy that renames an element into a breakout name inside a foreign root, drops an integration point under a nested root, or renames or drops the table around foreign parts, four random probe inputs are not fixed points on the first pass, as noted above.
• Once the input parser's context is unknown, the output tracker is not trusted on its own: an element there keeps HTML containment even where the output is foreign, as on main.
• The receiver with no policy below never learns whether its output entered a foreign root, so it still emits the item inside the root, as it did before. PolicyFactory is unaffected.
• The lexer reads <textarea> as RCDATA inside <svg> too, where a browser reads markup; the content comes out escaped, so it fails closed. Not this item.

Part of #492 (item 1)

🤖 Generated with Claude Code