Extensible parser / AST based on swc_*_parser / swc_*_ast

[kwonoj]

Note: This is neither formal RFC nor actual roadmap. Simply filiing for the tracking purpose for the case if we really need this.

For the transform side (ecma only as of today) it is possible to apply custom transform before / after as the chain of core transform. There is no similar thing for the parsing phase in contrast, it is not easily doable using existing parser infrastructure to expand / support custom behavior.

Note swc_* itself supporting only spec'ed behavior (ecma, html, css) is very deliberate decision. SWC core itself will never officially, publicly support non-standard syntax regardless of SWC core itself have those features or not. (Means, even if swc/core supports some non-standard features one day, that won't be under semver compliant and user is on their own to use it).

But it is a different story if someone wants to create a custom behavior on top of swc. There are couple of existing references in the world where famous framework with custom syntax, like vue, svelte, etcs. If someone want to constrct an AST understand those based on SWC's ast, there's no simple way to do it.

Potentially SWC may open up some extensibility to existing architecture to support those usecase. Users may able to attach, or extend, a specific part of parser / AST to conform to those custom behavior, then eventually pass it back to swc's core to run remaining transforms. This includes several questions, however, since this is not a RFC with details, so we'll not try to address in deeply. Also lastly, this doesn't mean to support runtime plugin support as similar to plugin does

[kwonoj]

/cc @kdy1 based on our casual chat discussion.

High-level idea suggested at the moment was defining an extensible trait for the parser, and make parser, lexer, token to support generics. This would be a fairly large change.

[kdy1]

I'm not sure how many patches actually we will need, so these are just some thoughts.

---

I think a custom AST will require too many generics, so it should be at most custom parser/lexer/token extension at least in the initial release.

In other words, the parser plugin can create a custom token but should emit the standard ES AST. So the plugin should also handle transforms. This is somewhat different from previous design decisions of SWC, but actually, this is required to prevent API regression of other rust-side users.

If we want to support custom AST, it should be one of

• Box<dyn Any> in various types, but it's impractical due to custom transform plugins
• All nodes are generic, while passing only standard ES to transform plugins

But I don't think this is something we can even try. All AST node types being generic is way too much regression.

Instead, parser / lexer plugin can lex / parse / transform custom syntax into the standard AST. This is harder than parsing as a custom AST and transforming it, but some tools like sucrase or esbuild do it. They do it for performance, but it's not important here. They prove that lexer/parser level transforms are possible.