### 🔎 Search Terms `getChildren`, `addSyntheticNodes`, missing token, type arguments, `<<`, `reScanLessThanToken`, `LessThanLessThanToken`. ### 🕗 Version & Regression Information - This is the behavior in every version I tried ### ⏯ Playground Link _No response_ ### 💻 Code ```js const ts = require('typescript'); const code = `type Bar = ReturnType<<T>(x: T) => number>;`; const sf = ts.createSourceFile( 'a.ts', code, ts.ScriptTarget.ESNext, /*setParentNodes*/ true, ); (function print(node, depth = 0) { console.log( ' '.repeat(depth) + `${ts.SyntaxKind[node.kind]} [${node.pos},${node.end}]` + ` ${JSON.stringify(code.slice(node.getStart(sf), node.end))}`, ); for (const child of node.getChildren(sf)) { print(child, depth + 1); } })(sf); ``` ### 🙁 Actual behavior The `TypeReference`'s children skip the `<` at `[21, 22]` entirely - there is a hole between the `Identifier` ending at 21 and the `SyntaxList` starting at 22: ```text TypeReference [10,42] "ReturnType<<T>(x: T) => number>" Identifier [10,21] "ReturnType" SyntaxList [22,41] "<T>(x: T) => number" <-- gap at [21,22] GreaterThanToken [41,42] ">" ``` The token for first `<` is absent. Walking `getChildren()` recursively to collect leaf tokens yields a token stream that does not cover the whole source text. ### 🙂 Expected behavior A `LessThanToken` at `[21, 22]`, so that the children of `TypeReference` are contiguous and the leaf tokens cover the source. TypeScript's own AST is correct and only `getChildren()` disagrees with it. Babel emits this token. ### Additional information about the issue The parser behaves correctly. It calls `reScanLessThanToken()`, splits the `<<`, and produces a correct tree with correct positions. The problem is in the services layer: `NodeObject.getChildren()` -> `createChildren()` -> `addSyntheticNodes()` in `src/services/services.ts`. The parser does not retain a child node for a type argument list's `<`, so `getChildren()` recovers such tokens by re-scanning the text in the gaps between the children it did retain - and discards any token that overruns the gap: ```js function addSyntheticNodes(nodes, pos, end, parent) { scanner.resetTokenState(pos); while (pos < end) { const token = scanner.scan(); const textPos = scanner.getTokenEnd(); if (textPos <= end) { // <-- 23 <= 22 is false, so the `<` is dropped // ... nodes.push(createNode(token, pos, textPos, parent)); } pos = textPos; // ... ``` Here it is called with `pos = 21`, `end = 22`. The scanner returns a single `LessThanLessThanToken` ending at 23, `23 <= 22` is false, the token is thrown away, `pos` advances to 23, and the loop exits with the gap unfilled. There's an asymmetry with `>`. The scanner emits `>` one character at a time and only _merges_ on re-scan (`reScanGreaterToken`), so `Foo<Bar<Baz>>` is unaffected. `<` is the only direction where a plain scan produces something **longer** than the parser wanted. `addSyntheticNodes` scans with no knowledge of the re-scan decisions the parser made, so any token the parser split can be lost this way. ### Suggested fix In `addSyntheticNodes`, rather than discarding an overrunning token, clip it to `end` or re-scan it - the scanner already exposes `reScanLessThanToken()` for this purpose. ### Scope In practice `<<` is the only sequence that triggers this, because `reScanLessThanToken` is the only splitting re-scan reachable outside JSDoc. It is called from exactly two places, both opening a type argument list - `parseTypeArgumentsOfTypeReference` and `parseTypeArgumentsInExpression`. The other splitting re-scans (`reScanAsteriskEqualsToken`, `reScanQuestionToken`) are JSDoc-only. Affected forms include: ```ts type X = ReturnType<<T>(x: T) => number>; // `<` at [19,20] missing type Y = ReturnType <<T>(x: T) => number>; // `<` at [20,21] missing type Z = ReturnType/* c */ <<T>(x: T) => number>; // `<` at [27,28] missing const a = foo<<T>(x: T) => T>(); // `<` at [13,14] missing ``` ### Relation to existing issues Issue for the corresponding _parse error_ was #23996, fixed in TypeScript 3.3.1 by #26653. That PR changed `src/compiler/parser.ts` and `src/compiler/scanner.ts` only, so the code now _parses_ - but `src/services/services.ts` was not given the same treatment, which is why `getChildren()` still loses the token. #47410 tracks the parser positions #26653 did not reach. This issue is the services-side counterpart. ### Downstream impact typescript-eslint builds its ESTree token list by walking `getChildren()`, so the token is missing from every consumer's `SourceCode` - any lint rule using `getTokenBefore`, `getTokensBetween`, or similar sees a different token stream depending on whether the project uses the TypeScript or Babel parser. https://github.com/typescript-eslint/typescript-eslint/issues/12820 ### Contribution I'd be very happy to make a PR to fix the JS code. However, I assume this bug may also manifest in TS 7 (Go), and I don't know what policy is on "old" TS and "new" TS diverging. Or maybe this doesn't apply, as TS 7 don't yet expose tokens to user code, so whatever it does internally is not observable?