-
Notifications
You must be signed in to change notification settings - Fork 58
Making the Morphir Elm frontend Incremental
The Morphir Elm frontend is a crucial part of the Morphir ecosystem. It allows us to build business models using the syntax and tools provided by the Elm programming language and turn them into Morphir IR. It is implemented as a command-line tool that reads a set of Elm source files from the filesystem, parses them and transpiles them into a single JSON file that is a serialization of the Morphir IR data structure. The tool itself is mostly implemented in Elm with a small JavaScript wrapper that focuses on the I/O part (which is not supported in Elm by design to keep it completely side-effect free). The JavaScript code then runs in Node.js and exposed as a command in an NPM package (morphir-elm make).
The initial implementation focused on functional correctness and completeness, and we paid little attention to other, non-functional requirements like performance and resource efficiency. Thanks to functional programming even without explicitly focusing on performance the implementation scaled really well to the bulk of our use-cases. In certain edge-cases though, we ran into some slowness, especially around type inference on large functions which made us consider improving the process.
There are some obvious inefficiencies in the current process:
- The tool always starts from scratch and processes every Elm file in the project even though in most cases only a few of them will change from one run to the next.
- This means that we are potentially doing several magnitudes more work than we need. At an extreme, if no sources changed, we could avoid doing any processing.
- Elm sources are always parsed, even if they are not reachable through exposed modules and therefore won't be part of the resulting JSON.
- There is no way to parallelize the current process since it's implemented as a single function.
Based on the above we decided to invest in refactoring the frontend to make it scale better and decided that the biggest RoI is on making it incremental. Incremental means that we will consider previous runs and reuse as much as we can from the previous IR.