Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix resolution of for-generator vars within iteratee #747

Closed
wants to merge 1 commit into from

Conversation

bioball
Copy link
Contributor

@bioball bioball commented Oct 29, 2024

This is one attempt to fix #741. The approach here is to wrap the iterable in a new root node; that is:

res {
  for (i in List(1, 2)) {
//          ^^^^^^^^^^  now a root node
    i
  }
}

We then store any existing for-generator vars as auxiliary slots inside this root node.

I'm not entirely happy with this approach; one quirk here is that we are creating a deeper stack frame, but cloning the parent stack frame. We need to do this so that we can resolve variables that might be on the outer frame.

Going deeper: the reason we need to copy outer frame slot values is because our lexical scope is all based off of VmObjectLike; we don't have a mechanism for resolving a variable on the outer frame if there is no VmObject to attach a frame to.

However, we don't have this pattern anywhere in the codebase. It's a one-off and feels quite hacky.

Also, creating a new root node adds more interpreter overhead. But the cost there seems palatable; it's unclear to me how much this actually hurts our real world performance.

I can think of two alternatives to this approach:

  1. Deep-force the iterable prior to executing each loop. However:
    • This will introduce other regressions (deep-forcing might throw, but for-generators today may not necessarily execute those paths).
    • This is likely terrible for performance (think: iterating over a glob import)
  2. Resolve variables at parse-time, within AstBuilder. I think we want to do this eventually, but it's a big effort.

Fix an issue where a lazy member within a for-generator or spread
breaks resolution.
@odenix
Copy link
Contributor

odenix commented Oct 29, 2024

Resolving variables at parse time would also help with meta-programming (Truffe AST transformations). In my PBT prototype, I had to resolve variables on my own, which was problematic.

@bioball
Copy link
Contributor Author

bioball commented Oct 30, 2024

Yeah; parse-time variable resolution is also needed for flat member syntax. At runtime, we don't have enough information to know that this shouldn't resolve to the middle bar in:

foo.bar.baz = bar

@bioball
Copy link
Contributor Author

bioball commented Oct 30, 2024

Elaborating on the parse-time resolution approach:

I think the better fix here is: if we know that the iteratee references a for-generator variable, produce node that executes those paths when executing the for-generator.

I'm not going to merge this; not totally convinced this is the correct solution.

However, we're adjusting the typechecking to prevent this bug from introducing regressions, due to lazy mapping/listing typechecks: #752

@bioball bioball closed this Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Late-bound values of iteratees within nested for/spread fail to resolve for-generator variables
2 participants