Fix scanning issues related to Unicode escapes in RegExp group names and refactor identifier scanning - #63996
Open
graphemecluster (graphemecluster) wants to merge 2 commits into
Conversation
Copilot started reviewing on behalf of
graphemecluster (graphemecluster)
August 25, 2026 07:04
View session
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes Unicode-escape scanning and normalization across identifiers and RegExp group names.
Changes:
- Unifies standard, JSX, JSDoc, private, and RegExp identifier scanning.
- Supports leading escapes and surrogate-pair escapes in RegExp group names.
- Adds compiler tests and updated baselines.
Reviewed changes
Copilot reviewed 47 out of 53 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
tsc/internal/scanner/scanner.go |
Refactors identifier scanning and escape handling. |
tsc/internal/scanner/regexp.go |
Uses RegExp-aware identifier scanning. |
tsc/testdata/tests/cases/conformance/jsx/unicodeEscapesInJsxtags.tsx |
Extends JSX escape coverage. |
tsc/testdata/tests/cases/compiler/unicodeEscapesInNames03.ts |
Adds property and type-name cases. |
tsc/testdata/tests/cases/compiler/unicodeEscapesInNames01.ts |
Adds multi-escape identifier cases. |
tsc/testdata/tests/cases/compiler/unicodeEscapesInJSDoc.ts |
Tests escaped JSDoc names. |
tsc/testdata/tests/cases/compiler/regularExpressionGroupNameUnicodeEscapes.ts |
Tests RegExp group-name escapes. |
tsc/testdata/tests/cases/compiler/extendedUnicodeEscapeSequenceIdentifiers.ts |
Tests multiple extended escapes. |
tsc/testdata/baselines/reference/conformance/unicodeEscapesInJsxtags.types |
Updates JSX type baseline. |
tsc/testdata/baselines/reference/conformance/unicodeEscapesInJsxtags.symbols |
Updates JSX symbol baseline. |
tsc/testdata/baselines/reference/conformance/unicodeEscapesInJsxtags.js |
Updates JSX emit baseline. |
tsc/testdata/baselines/reference/conformance/unicodeEscapesInJsxtags.errors.txt |
Updates JSX diagnostics. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=esnext).types |
Adds ESNext type baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=esnext).symbols |
Adds ESNext symbol baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=esnext).js.map |
Adds ESNext source maps. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=esnext).js |
Adds ESNext emit baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=es2015).types |
Adds ES2015 type baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=es2015).symbols |
Adds ES2015 symbol baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=es2015).js.map |
Adds ES2015 source maps. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames03(target=es2015).js |
Adds ES2015 emit baseline. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=esnext).types |
Updates ESNext types. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=esnext).symbols |
Updates ESNext symbols. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=esnext).sourcemap.txt |
Updates ESNext source-map record. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=esnext).js |
Updates ESNext emit. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=es2015).types |
Updates ES2015 types. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=es2015).symbols |
Updates ES2015 symbols. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=es2015).js |
Updates ES2015 emit. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInJSDoc.types |
Updates JSDoc types. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInJSDoc.symbols |
Updates JSDoc symbols. |
tsc/testdata/baselines/reference/compiler/unicodeEscapesInJSDoc.js |
Updates JSDoc emit. |
tsc/testdata/baselines/reference/compiler/shebangError.types.diff |
Adds an unregistered diff artifact. |
tsc/testdata/baselines/reference/compiler/shebangError.types |
Updates shebang recovery types. |
tsc/testdata/baselines/reference/compiler/shebangError.js.diff |
Adds an unregistered diff artifact. |
tsc/testdata/baselines/reference/compiler/shebangError.js |
Updates shebang recovery emit. |
tsc/testdata/baselines/reference/compiler/shebangError.errors.txt.diff |
Adds an unregistered diff artifact. |
tsc/testdata/baselines/reference/compiler/shebangError.errors.txt |
Updates shebang diagnostics. |
tsc/testdata/baselines/reference/compiler/regularExpressionGroupNameUnicodeEscapes.types |
Adds RegExp type baseline. |
tsc/testdata/baselines/reference/compiler/regularExpressionGroupNameUnicodeEscapes.symbols |
Adds RegExp symbol baseline. |
tsc/testdata/baselines/reference/compiler/regularExpressionGroupNameUnicodeEscapes.js |
Adds RegExp emit baseline. |
tsc/testdata/baselines/reference/compiler/regularExpressionGroupNameUnicodeEscapes.errors.txt |
Adds RegExp diagnostics. |
tsc/testdata/baselines/reference/compiler/manyCompilerErrorsInTheTwoFiles.types.diff |
Adds an unregistered diff artifact. |
tsc/testdata/baselines/reference/compiler/manyCompilerErrorsInTheTwoFiles.types |
Updates recovery types. |
tsc/testdata/baselines/reference/compiler/manyCompilerErrorsInTheTwoFiles.js.diff |
Adds an unregistered diff artifact. |
tsc/testdata/baselines/reference/compiler/manyCompilerErrorsInTheTwoFiles.js |
Updates recovery emit. |
tsc/testdata/baselines/reference/compiler/manyCompilerErrorsInTheTwoFiles.errors.txt |
Updates recovery diagnostics. |
tsc/testdata/baselines/reference/compiler/extendedUnicodeEscapeSequenceIdentifiers.types |
Updates extended-escape types. |
tsc/testdata/baselines/reference/compiler/extendedUnicodeEscapeSequenceIdentifiers.symbols |
Updates extended-escape symbols. |
tsc/testdata/baselines/reference/compiler/extendedUnicodeEscapeSequenceIdentifiers.js |
Updates extended-escape emit. |
Files not reviewed (2)
- tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=es2015).js: Generated file
- tsc/testdata/baselines/reference/compiler/unicodeEscapesInNames01(target=esnext).js: Generated file
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
| @@ -0,0 +1,12 @@ | |||
| --- old.shebangError.types | |||
| @@ -0,0 +1,9 @@ | |||
| --- old.shebangError.js | |||
| @@ -0,0 +1,26 @@ | |||
| --- old.shebangError.errors.txt | |||
| @@ -0,0 +1,23 @@ | |||
| --- old.manyCompilerErrorsInTheTwoFiles.types | |||
| @@ -0,0 +1,15 @@ | |||
| --- old.manyCompilerErrorsInTheTwoFiles.js | |||
Contributor
Author
|
All checks passed, so I am leaving the diff files here. If they are to be removed, feel free to do so on my behalf. |
Member
|
We definitely don't want the diffs, we just don't have anything that checks for extra baselines (my mistake for not retaining that) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per microsoft/typescript-go#4881 (review), this is a port of the unmerged #61042, which fixes that leading Unicode escapes and surrogate pair escapes are not parsed and normalised in regular expression group names and result in TS1514.
By this fix, identifier scanning is refactored and unified across normal, private, JSX, JSDoc identifiers and RegExp group names.
The first commit is a faithful port of Fix scanning issues related to Unicode escapes in identifiers #61042 including the 6 test cases there. The issue detailed in Text between Unicode escapes within an identifier is skipped #61043 (which the old PR also fixes) does not exist in TypeScript Go, and some parts of the old PR are already implemented or are no longer relevant, so not everything is ported.
The second commit contains some more refactoring (centralising surrogate pair logic) and a tiny optimisation (?) for private identifiers.
Fixes #63995
Closes #61043