Replies: 28 comments 66 replies
-
Yep, very similar to what I'm thinking. I drew this a while back |
Beta Was this translation helpful? Give feedback.
-
I also sketched a desktop application that looks like the following way (maybe it's a bit overkill but it's the first version and it can be simplified and improved): Sources are available here: DAGE I suggest using the following grammar processing stages:
The first stage can be implemented either on client or on server, other staged can be implemented only on server because they requires JVM runtime. The last stage can be implemented on server, but also on client for JS runtime. I suggest using Kotlin for the service implementation since it has good compatibility with Java and it can be compiled both to JVM or to JavaScript (Kotlin multiplatform). |
Beta Was this translation helpful? Give feedback.
-
any chance we can use kotlin to get a full js-only version of antlr tool? |
Beta Was this translation helpful? Give feedback.
-
There are only a few online parser tools, which is a sad state we are in. I was writing a web-based Antlr tool, but I haven't been able to get far because of too many other higher priorities. I would try to avoid writing an Antlr tool running on a server, and instead try to use WebAssembly from the Java classes--which is what I was trying to do. I would also highly recommend not using the ancient Antlr4, Antlr4.Runtime, Antlr4.ToolGenerator code. It does not support current grammars. |
Beta Was this translation helpful? Give feedback.
-
Why would you want to go through all that trouble ? Just run the tool on AWS Lambda.
What we need in the browser is a grammar editor with syntax checking, but that’s a small portion of the tool, especially if the latter is delegated to the backend
Envoyé de mon iPhone
… Le 21 mai 2022 à 21:21, Terence Parr ***@***.***> a écrit :
any chance we can use kotlin to get a full js-only version of antlr tool?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
|
Beta Was this translation helpful? Give feedback.
-
My main interest in a web based tool or even a local single page app is (1) debugging grammars w/o needing intellij and (2) allowing collaboration so people can submit bugs with a grammar, input, and associated state from a parse. |
Beta Was this translation helpful? Give feedback.
-
Question.
|
Beta Was this translation helpful? Give feedback.
-
I just tried implementing a webpage that would operate like the antlr plugin for intellij/jetbrains. Dynamically loading code is easy now with the import statement but I could not get it to pull in new code; I think with some tricks I can get it to pull in new code but it would be redefining functions and I think the heap would just keep getting bigger as the old versions of the code would not get collected. There isn't a parser interpreter for JS like in some of the other target and so I can't pull in the new parser as just data. :( Ideas are welcome. At the moment, I need a server that runs the Java antlr tool to generate parsers. Also because I cannot reload those parsers, I need that tool to also do the parsing and send the results back to the webpage. unpleasant but doable. Suggestions welcome. JS/TS seems like a reasonable language but the ecosystem import/requires/node-vs-browser etc... Seems to be a God awful mess. |
Beta Was this translation helpful? Give feedback.
-
In fact, most of the necessary pieces for this project exist already.
Obviously all these parts need adjustments to fit into this new ANTLR Playground tool, but at least they are all web-ready. No need to invent the wheel again. The main challenge IMO is the generation of the interpreter data, as it requires a running ANTLR tool. For now this could be done as mentioned here already, maybe limiting the number of executions per time unit (to avoid high costs). My TS port of ANTLR4 could help out here later to have this run entirely in the browser (using web workers for time consuming tasks). But it's currently in a very early state, so this will take some time until it's ready. |
Beta Was this translation helpful? Give feedback.
-
I expect you’ll find it very hard to have a decently responsive UI without delegating parsing, analysis and checking to a worker.
That would remove entirely the constraint you are mentioning when selecting a UI framework.
Also, have cou considered ace as an alternative to monaco ?
… Le 13 juin 2022 à 15:23, Mike Lischke ***@***.***> a écrit :
With that decision you are constraining yourself very much. But after all this is your idea. Good luck!
—
Reply to this email directly, view it on GitHub <#3717 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJD3FJD6BTIWF4AMBSDVO4Y6VANCNFSM5WCO7PGQ>.
You are receiving this because you commented.
|
Beta Was this translation helpful? Give feedback.
-
What a great discussion we have here! We are also rich in possible implementation strategies! haha. We also have at least three potential proj owners/leaders: me, @KvanTTT, and @mike-lischke. PossibilitiesMike: with your VSCode plugin, could we turn that into an isolated application or webpage that contacted an antlr-tool-service running either locally or remotely to process grammar and generate interpreter data? As @KvanTTT pointed out, with all of the dependencies converting the complete tool chain to JavaScript/TS could be problematic. I'm guessing that launching a local service is the simplest approach to extract interpreter data. I looked at the vscode/monaco stuff and that would be a huge learning task for me so I'm guessing I'm not gonna go down that path personally, but if somebody wants to launch this and use VSCode I'm totally happy with that. In the end, I think we should have a very modest goal, which makes it a fairly simple problem. We're not trying to build a full development environment. In my view, this is really a debugger and learning tool:
Grammar sharingAnother component is the idea of sharing 40 bugging or education purposes. In other words, someone finds a problem with the parse or wants to demonstrate some thing to another person. Just like link shorteners, people could click a button to get Sharing link, but this would require a publicly visible server that saved grammars and input so that others could launch using that link. My natural inclination at this point would be to simply start building something now that we have some understanding the primary roadblocks. I'm not saying I should necessarily be this person, but my first whack at the problem would be to simply create a python flask-based server that installed java/antlr (using https://github.com/antlr/antlr4-tool) then answered calls to process grammars. Next, a webpage to send grammars to the server for processing upon keystroke or button push. It wouldn't be pretty but it would be running within a few days. That would give us something concrete to evaluate and discuss. |
Beta Was this translation helpful? Give feedback.
-
There's VS Code for the web, which even can use some of the desktop VS Code extensions, Unfortunately, mine is not among them, as it requires Node.js functionality (processes, file system).
The only dependency is Java. That ANTLR4 requires some ANTLR3 and ST4 classes is not a blocking stone. Those classes belong to the same problem class like converting ANTLR4 itself to TS.
A python app can open a web socket on the host of the web application and listen for commands coming from the web interface (which connects to this socket). It can handle the Java invocation to generate interpreter data and it can store the snapshots on disk which are needed for the web links for the playground. It can even provide access to all existing grammars (running get commands against Github) and provide enumeration for that to show a grammar tree in the UI (if wanted). Thinking this further, the python server could handle everything what's done in the backend part of my extension: interpreter data generation, error checking, grammar file handling (interdependencies), helper functions for the UI providers (symbol retrieval, code completion, RRD generation, ATN graph generation, sentence generation, debugging) etc. Check the facade.ts file for all the features. This file is the interface between FE and BE. In the playground version this file would instead generate websocket requests and convert the received data from the websocket to something the FE can process. With this approach you would never need a TS version of ANTLR4. |
Beta Was this translation helpful? Give feedback.
-
I just remembered the cool debugger built by @jean-bovet that we had for antlrworks (antlr3): I think that would be very useful to have as well. The way it worked was a profiling/tracing parser listener tracked every action by the parser and then we used a time traveling debugger idea to move forward and backwards during parsing. |
Beta Was this translation helpful? Give feedback.
-
Crap. Trying to learn js/ts and cobble together bits from antlr4ts / mike's stuff seems a sloooow road. Been converting ts to js to use js target but no doubt my efforts will take weeks to start running. Maybe I should try pure antlr4ts and see if I can get working browser? |
Beta Was this translation helpful? Give feedback.
-
OK now we're getting somewhere... See https://github.com/antlr/antlr4-lab With a simple web form like this: the new JSON parse tree stuff sends back information from the server we can use to Display information about the parse: |
Beta Was this translation helpful? Give feedback.
-
Working on a prototype...When you hit “RUN”, server gets grammar/input/startrule and returns big wad of results/errors etc… I quietly rebuild input window text with items to add tool tips for hover info on each token in input. Does not affect input text display :) works like magic. MUUUUCH simpler than working with intellij. screenshot shows hover tooltip when cursor is over “able”. |
Beta Was this translation helpful? Give feedback.
-
@parrt I played a bit with Github.dev today and I think I can make my ANTLR4 extension ready for the web. Users would then use e.g. https://github.dev/antlr/grammars-v4 to open any grammar and play with it. Not fully what @KvanTTT had in mind, but requires almost no work from your side. Just a thought... |
Beta Was this translation helpful? Give feedback.
-
Better buttons, shows token preview. Also got show/hide profile button: Profile info and tree: |
Beta Was this translation helpful? Give feedback.
-
You can try http://lab.antlr.org if you like :) |
Beta Was this translation helpful? Give feedback.
-
I'm interested in multiple parses from an ambiguous grammar. The idea is that if I build up a grammar bottom up from examples of Or maybe generate rules that are special cases, and only afterward, I like http://lab.antlr.org because it lets me "doodle" with a grammar Are my goals too hard for the tool as currently implemented ? |
Beta Was this translation helpful? Give feedback.
-
Got SVG trees in now. See antlr/antlr4-lab#6 |
Beta Was this translation helpful? Give feedback.
-
Testing new @antlr/core tag. Did you see this Tom? |
Beta Was this translation helpful? Give feedback.
-
Hi All, I prototyped some UI in antlr4-lab to add a control for loading a grammar from grammars-v4. If anyone is interested, you can see what it looks like over here. Comments welcomed. |
Beta Was this translation helpful? Give feedback.
-
Hi,On top of Ken’s proposal, various ideas that come to mind:- upload a grammar from an existing g4- download a grammar as g4- provide a ‘combined’ tab to avoid the probable confusion re Lexer/parse with newbiesEnvoyé de mon iPhoneLe 4 oct. 2022 à 12:23, Ken Domino ***@***.***> a écrit :
Hi All, I prototyped some UI in antlr4-lab for adding a control for loading a grammar from grammars-v4. If anyone is interested, you can see what it looks like over here. Comments welcomed.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Here's what that grammars-v4 select now looks like: As I mentioned elsewhere, I can flip through grammars and check out at the performance profile for the input really quickly. So, this could be a really useful tool. Right now there's one select control for selecting a grammar plus one input. I'm wondering about whether there should be two select controls, one select for the grammars like "abb", "abnf", "cpp", etc., positioned by the "lexer/parser" tabs, and a second select for the various examples in the examples/ directory for the grammar, positioned over by the "Input" tab on the right-hand side. With all these great ideas, we're going to run out of real estate. BTW, does anyone know how zoom over the parse tree drawing is supposed to work? I have a standard Windows desktop with two mouse buttons and one wheel. |
Beta Was this translation helpful? Give feedback.
-
May I suggest that the grammar selector would fit more naturally above the lexer/parser tabs ?
… Le 4 oct. 2022 à 19:35, Ken Domino ***@***.***> a écrit :
Here's what that grammars-v4 select now looks like:
<https://user-images.githubusercontent.com/11389451/193885307-3e9079b0-cc06-4f43-9886-3721a45693d6.png>
As I mentioned elsewhere, I can flip through grammars and check out at the performance profile for the input really quickly. So, this could be a really useful tool.
Right now there's one select control for selecting a grammar plus one input. I'm wondering about whether there should be two select controls, one select for the grammars like "abb", "abnf", "cpp", etc., positioned by the "lexer/parser" tabs, and a second select for the various examples in the examples/ directory for the grammar.
With all these great ideas, we're going to run out of real estate.
BTW, does anyone know how zoom over the parse tree drawing is supposed to work? I have a standard Windows desktop with two mouse buttons and one wheel.
—
Reply to this email directly, view it on GitHub <#3717 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJCMRQAKOX4YICQ4QALWBRTEJANCNFSM5WCO7PGQ>.
You are receiving this because you commented.
|
Beta Was this translation helpful? Give feedback.
-
Jupyter maybe a choice but the antlr-lab is more convenient. |
Beta Was this translation helpful? Give feedback.
-
There are a lot of services for online regex writing and testing: https://regex101.com/, https://regexr.com/, https://www.regextester.com/, and so on. They allow everyone to write and test any regular expression very easily.
On the other hand, there are services that allow parsing, analyzing, and compiling specified programming languages: https://sharplab.io/, https://astexplorer.net/, https://godbolt.org/
But there are no services that allow creating parsers right from the browser and based on grammar.
I suggest creating such a service based on ANTLR parser generator.
The service will have at least two text fields. The left one contains an editor for grammar. The right one contains an editor for parse input. Also, yet another field can display a parse tree. The service will allow grammar editing and obtaining result parsers in different languages.
If implemented such a service may provide the following benefits:
By the way, a similar service exists but for binary parsing: https://ide.kaitai.io/ It would be great to have a service for text parsing!
A much simpler but also useful idea is about HTTP API. Everyone can just get generated parser by sending request and grammar to the cloud grammar generator.
Beta Was this translation helpful? Give feedback.
All reactions