wit feature request: add explicit item delimiters

[morigs]

I'd like to suggest adding explicit delimiters between interface/world items. For example, this could be semicolons:

interface example {
  a: func();
  b: func() -> ();
  c: func() -> t;
  type t = u32;
}

Reasoning:
I was experimenting with textmate grammars for WIT (tmLanguage is used by vscode) and discovered that it's difficult to add full-featured syntax highlighting because of tm limitations. In order to add "scope" to some part of the document, tm has to know exactly where it starts and ends. This works for interfaces (they start with { and end with }), however it causes problems with interface children. For example, functions can span multiple lines (so we can't use \n as end), they can have return type missing or specified without parenthesis (so we can't use ')').
Proto format uses semicolons for this.

There are workarounds: for instance it's possible to ignore meta and other conventional scopes and highlight keywords only instead. However, this will make writing wit files uncomfortable :(

VS Code supports "semantic tokens" which provides better highlighting for lsp-enabled languages. However, it's difficult to say if this is enough without actually implementing both tm and lsp

[alexcrichton]

This seems reasonable to me to add. I think the original inspiration for lack of semicolons was TypeScript buat making it easier on tools seems ok to me at least. That being said I don't have a ton of strong opinions about specific syntax and others may have more thoughts here too.

[lukewagner]

I've also been quietly worried that our lack of end-of-statement-delimiter may one day come back to bite us in terms of grammatical ambiguities. See also JS Automatic Semicolon Insertion which is a real nightmare.

[alexcrichton]

I've posted #249 as an example of implementing this along with a rollout plan which hopefully won't cause too much disruption from this.

[bashor]

Couldn't we avoid introducing a delimiter if every declaration inside the interface started with a keyword?

[bashor]

Isn't it better to use the same delimiter everywhere (inside interface, record, variant, enum, and flags)?

[alexcrichton]

From a grammatical perspective I don't think semicolons are required today to avoid ambiguities or anything like that, so in that sense I don't believe that semicolons are required. From a "it's easier to parse" and an "it's more extensible" perspective, however, having delimiters like semicolons I believe makes it easier to do syntax highlighting and quick-and-dirty parsing in addition to providing more flexibility in terms of grammar to add in the future.

For different delimiters, that's a good point! It indeed could be possible to use ; instead of , within the constructs you mention (apart from interface which already uses ;)

[leon-thomm]

If the component model achieves wide adoption, and interfaces will be read many times, this decrease in readability not technically necessary (and this is a lot of redundancy) feels like a step backwards.

If delimiters in the grammar are likely to be necessary at some point anyway then I'm with you, but I would not think so since clearly enough languages don't need them, some of which have excellent vscode integration.