Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(linter): typescript-eslint/no-floating-promises #2912

Closed
wants to merge 24 commits into from

Conversation

valeneiko
Copy link
Contributor

@github-actions github-actions bot added A-linter Area - Linter A-cli Area - CLI labels Apr 7, 2024
@Boshen
Copy link
Member

Boshen commented Apr 8, 2024

OMG this is amazing, fingers crossed for communicating with the ts server, as discussed in #2855

Copy link

codspeed-hq bot commented Apr 8, 2024

CodSpeed Performance Report

Merging #2912 will degrade performances by 3.74%

Comparing valeneiko:no-floating-promises (9e8ba4d) with main (d87cf17)

Summary

⚡ 1 improvements
❌ 1 regressions
✅ 34 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main valeneiko:no-floating-promises Change
parser_napi[checker.ts] 230.3 ms 239.2 ms -3.74%
sourcemap[react.development.js] 21 ms 19.6 ms +6.82%

@Boshen
Copy link
Member

Boshen commented Apr 11, 2024

@valeneiko Such an amazing effort! Feel free to ping me with preliminary results!

@github-actions github-actions bot added the A-ast Area - AST label Apr 14, 2024
@valeneiko
Copy link
Contributor Author

@Boshen Almost there!

It now successfully detects a floating Promise in a simple file:

cargo run --bin oxlint -- --typecheck-plugin -A all -D no-floating-promises \
  --tsconfig ./crates/oxc_linter/fixtures/typecheck/tsconfig.json \
  ./crates/oxc_linter/fixtures/typecheck/sample.ts
async function funcAsync() {
  return Promise.resolve();
}

function func() {
  return 7;
}

export async function test() {
  func();
  funcAsync();
}

first_result

@Boshen
Copy link
Member

Boshen commented Apr 26, 2024

image

I tested this on cal.com, and it's working on multiple files!

It panic with

called `Result::unwrap()` on an `Err` value: CommandFailed("Error processing request. Cannot read properties of undefined (reading 'kind')\nTypeError: Cannot read properties of undefined (reading 'kind')\n    at isDeclarationNameOrImportPropertyName (/Users/bytedance/github/oxc/npm/oxc-typecheck/node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js:87886:25)\n    at getTypeOfNode (/Users/bytedance/github/oxc/npm/oxc-typecheck/node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js:85186:11)\n    at Object.getTypeAtLocation (/Users/bytedance/github/oxc/npm/oxc-typecheck/node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js:46305:23)\n    at Module.isPromiseArray (file:///Users/bytedance/github/oxc/npm/oxc-typecheck/dist/rules/no-floating-promises.js:4:26)\n    at noFloatingPromises::isPromiseArray (file:///Users/bytedance/github/oxc/npm/oxc-typecheck/dist/handlers.js:46:43)\n    at executeCommand (file:///Users/bytedance/github/oxc/npm/oxc-typecheck/dist/server.js:41:26)\n    at onMessage (file:///Users/bytedance/github/oxc/npm/oxc-typecheck/dist/server.js:26:48)\n    at Interface.<anonymous> (file:///Users/bytedance/github/oxc/npm/oxc-typecheck/dist/server.js:16:9)\n    at Interface.emit (node:events:518:28)\n    at [_onLine] [as _onLine] (node:internal/readline/interface:416:12)")

I think it's due to providing utf8 span positions instead if utf16, we don't need to worry about this as there is already a tracking issue.

Now onto performance: if performance of what we have right now is less or equal to @typescript/eslint, then we are all good!

@Boshen
Copy link
Member

Boshen commented Apr 26, 2024

@valeneiko Can you time our setup and compare to one or two repositories with no-floating-promises on? https://github.com/search?q=no-floating-promises&type=code

https://github.com/mermaid-js/mermaid seems like a good place to start: Finished in 41ms on 397 files with 89 rules using 12 threads.

If the timing is similar for files less than a thousand, then we can start cleaning things up and merge things!

@valeneiko
Copy link
Contributor Author

@Boshen I have tried to run lint on the repo you linked, but it fails with the exact same error. And it's cause is not just encoding differences.

Take a look at the AST of one of the files from the repo:

OXC makes a request for CallExpression at (58, 1758), but typescript has this node at (56, 1754). Looks like typescript includes leading newlines and spaces as part of the CallExpression span.

Do you have any suggestions for how we can better map between the two? I am thinking we could try a full path or just child indexes, but now quite sure how to get this from OXC.

On a side note, I finished the remaining parts of the lint rule.

@Boshen
Copy link
Member

Boshen commented Apr 27, 2024

@Boshen I have tried to run lint on the repo you linked, but it fails with the exact same error. And it's cause is not just encoding differences.

Take a look at the AST of one of the files from the repo:

OXC makes a request for CallExpression at (58, 1758), but typescript has this node at (56, 1754). Looks like typescript includes leading newlines and spaces as part of the CallExpression span.

Do you have any suggestions for how we can better map between the two? I am thinking we could try a full path or just child indexes, but now quite sure how to get this from OXC.

On a side note, I finished the remaining parts of the lint rule.

Ahh such a big blocker, where is typescript eslint do the conversion? It's probably in https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-estree/src/ast-converter.ts but I can't find it from a quick scan.

I need to measure the effort of going forward with this experiment with we want to do the conversion.

In the meantime, can you help out and just ignore the error / crash and let the whole thing complete so we can compare the total running time between typescript eslint and our approach?

It's gonna be a lot of balancing of tradeoffs 🥵

@valeneiko
Copy link
Contributor Author

Ahh such a big blocker, where is typescript eslint do the conversion?

Here is the conversion logic in typescript-eslint: https://github.com/typescript-eslint/typescript-eslint/blob/9c94b81b2560558fcc2d85dd644679a5dbf4a277/packages/typescript-estree/src/convert.ts#L757

If I understand it correctly, typescript eslint build their own AST by visiting the AST typescript produced.

In the meantime, can you help out and just ignore the error / crash and let the whole thing complete so we can compare the total running time between typescript eslint and our approach?

Ok. I'll find a way to ignore those errors for now.

@Boshen
Copy link
Member

Boshen commented Apr 27, 2024

If I understand it correctly, typescript eslint build their own AST by visiting the AST typescript produced.

Yeah I saw that too, but I can't find where it's changing the span positions :-(

Ok. I'll find a way to ignore those errors for now.

Thank you!

@valeneiko
Copy link
Contributor Author

valeneiko commented Apr 27, 2024

Yeah I saw that too, but I can't find where it's changing the span positions :-(

They don't need to. Instead of looking up nodes by spans, the just directly map the nodes, because they store which TS node they used to create each one.

...

I've pushed the commits to ignore the errors. And managed to run the rule against mermaid repo. I've modified their .eslintrc.cjs to only enable @typescript-eslint/no-floating-promises and ESLint runs in 3.2 seconds.

pnpm eslint --ext ".js,.ts,.jsx,.tsx" .

OXC with that single rule takes 90 seconds (using 24 threads), or 220 seconds (single threaded). There is obviously something wrong.

..\rust\oxc\target\release\oxlint.exe --typecheck-plugin -A all -D no-floating-promises
  • It could be that mapping errors lead us to type check different code paths.
  • Out lint rule implementeation is not 100% correct, so we may be calling TS more than necessary
    • I know at least 1 case: () => new Promise(). ESLint completely skips those, but we call typecheck and generate an error.
  • But more likely there is something wrong in our JS side implementation. It should not be taking this long.

@Boshen
Copy link
Member

Boshen commented Apr 28, 2024

But more likely there is something wrong in our JS side implementation. It should not be taking this long.

I agree, my guess is that typescript eslint is using some kind of caching technique or some API that we don't know about 😅

@valeneiko
Copy link
Contributor Author

valeneiko commented Apr 28, 2024

Result of running OXC in debug mode against mermaid with all the measurements:

                      #: average /   total
                  parse: 0.007ms /  0.225s
              stringify: 0.019ms /  0.624s
                   open: 167.9ms / 66.672s
                  close: 0.320ms /  0.127s
         isPromiseArray: 0.173ms /  2.703s
          isPromiseLike: 0.014ms /  0.226s
isValidRejectionHandler: 0.039ms /  0.000s
             getProgram: 0.015ms /  0.460s
         getTypechecker: 0.000ms /  0.008s
                getNode: 0.004ms /  0.114s
        channelOverhead: 0.070ms /  2.261s
                   idle: 0.071ms /  2.283s

A few observations:

  • Actual work is basically the same as typescript-eslint
  • We spend over 3 seconds on just communication overhead
    • We could lower it by not sending file content
    • We should pipeline the requests, so that TS is never idle
    • We could use something faster than JSON
  • We spend most of our time opening files in TS
    • This seems like an unnecassary step - we should look into how ESLint does it

@Boshen
Copy link
Member

Boshen commented Apr 28, 2024

A few observations:

I see these are optimizations that can be done later.

Performance wise we are on the correct path, which is really good news 👍 .

Now the last puzzle piece to solve is the mismatched AST nodes. Let's explore this for a few days before jumping to any conclusions.

@valeneiko
Copy link
Contributor Author

After looking at a few different examples, OXC and TS have both different structure and different spans. So mapping code is unavoidable.

One idea comes to mind:

  • Build a visitor, that walks the tree in the same order on both sides. Then we can assign each node an id, and look it up by id on both sides. Visitor implementation in TypeScript is only about 600 lines (here). So not too diffucult to map. And we can probably exploit the fact that nodes are going to be visited in the order they appear in the source file. And only assign the Ids to nodes that map nicely (at least to start with), since we only need to be able to resolve a subset of them anyway.

@valeneiko
Copy link
Contributor Author

After exploring it a bit more, I don't think my previous idea would be enough. At this point the only solution I see is to fully map the AST. But given the effort required to maintain it I am not sure it would be worth it: we don't gain anything perf wise by having an ability to write these rules in rust.

@Boshen
Copy link
Member

Boshen commented May 5, 2024

After exploring it a bit more, I don't think my previous idea would be enough. At this point the only solution I see is to fully map the AST. But given the effort required to maintain it I am not sure it would be worth it: we don't gain anything perf wise by having an ability to write these rules in rust.

😢 It's sad to see that you have spent so much time and energy on this but it's blocked by how typescript works. TypeScript doesn't offer linter plugins, and we are forced to do this whole round trip despite us willing to do all these hard work, but we'll also need to maintain this whole round trip.

I think at this point, we should just stop thinking about porting these rules from @typescript/eslint and start innovating our own set of rules that can help people catch bugs.

I mean, it was tried before https://github.com/microsoft/TypeScript/pull/53146 and it wasn't prioritized ...

If anyone is going to ask again about typing rules, I'll just point them to this PR :-(


So back to the drawing board.

Since we have cross-file linting working with --import-plugin https://oxc-project.github.io/blog/2024-05-04-import-plugin-alpha.html, should we start innovating with cross-file information, and also look into things like #[must_use]?

What do you think?

@valeneiko
Copy link
Contributor Author

Since we have cross-file linting working with --import-plugin https://oxc-project.github.io/blog/2024-05-04-import-plugin-alpha.html, should we start innovating with cross-file information, and also look into things like #[must_use]?

What do you think?

Sound like a logical next step. It does feel like re-implementing TypeScript in Rust, but maybe we only need a very small subset to get most of the way. The complex part would be symbol resolution and I think we should start with that.

I am not so sure about adding extra annotation to code, I would prefer enabling this rule instead: explicit-module-boundary-types. Then the cross-file step could easily mark if a function or method returns a promise. And even without the rule, most of those functions would be annotated with async already, and we could use that instead of the return type. But regarless of the approach, we would need to have an ability to resolve references to their corresponding symbols, so that we could look up annotations on them.

@Boshen
Copy link
Member

Boshen commented May 6, 2024

Let me organize this a bit tomorrow.

The complex part would be symbol resolution and I think we should start with that.

If the goal is still no-floating-promises, I think we can get it working first by marking exported async functions of a module as async in ModuleRecord

/// Local exported bindings
pub exported_bindings: FxHashMap<CompactStr, Span>,

I am not so sure about adding extra annotation to code, I would prefer enabling this rule instead: explicit-module-boundary-types.

I meant marking things with extra annotations, there is a system in rust where you can mark things with different things in compile time, #[must_use] is one of them.

Now that you have mentioned it, maybe we can leverage isolated-declarations? And just do linting under isolated-declarations 🤔 We prepare for the future 😆

https://devblogs.microsoft.com/typescript/announcing-typescript-5-5-beta/#isolated-declarations

@valeneiko
Copy link
Contributor Author

Now that you have mentioned it, maybe we can leverage isolated-declarations?

Yes, this is exactly what I meant. But since that feature is not even out in TS, explicit-module-boundary-types could still make sense, and achieves the same result.

If the goal is still no-floating-promises, I think we can get it working first by marking exported async functions of a module as async in ModuleRecord

That could be a good start. I was mostly concerned about complexity of doing it for cases other than named import of a function:

// foo.ts
export function foo() {}
export const obj = { foo() {} };
export class Cls { foo() {} }

// -- default / namespace import
import mod from './foo';
mod.foo(); // <-- How do we resolve foo?

// -- object properties
import {obj} from './foo';
obj.foo(); // <-- How do we resolve foo?

// -- class fields
import {Cls} from './foo';
const x = new Cls();
x.foo(); // <-- How do we resolve foo?

@Boshen
Copy link
Member

Boshen commented May 8, 2024

I feel said about closing this PR :-( Thank you so so so much for working on this, I learned a lot.

Continue in #3105 (comment)

@Boshen Boshen closed this May 8, 2024
@Boshen
Copy link
Member

Boshen commented May 11, 2024

image

Source: https://speakerdeck.com/unvalley/exploring-type-informed-lint-rules-in-rust-based-linters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ast Area - AST A-cli Area - CLI A-linter Area - Linter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants