Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reliably detect and deforest list-like operations on pure values #157

Open
countvajhula opened this issue Jan 26, 2024 · 2 comments
Open
Labels

Comments

@countvajhula
Copy link
Collaborator

countvajhula commented Jan 26, 2024

How can we identify list-like operations done on values? See A Core Performance Issue Identified.

This would allow us to leverage the existing deforestation pipeline for pure values transformations.

We will likely need to generalize the stream implementation to support multiple input and output values at each step (in addition to zero or one). It should be usable in pipelines like (~> (-< (>< f) (>< g)) ...)

It's possible that a multi-valued stream implementation would be less performant than a single-valued one, but if it performs better than using pure values directly, then that is a worthwhile optimization. We need not use the same stream implementation for values as we do lists.

It may also be possible to extend stream fusion to other values-based combinators (in addition to ~>) like -<, ==, ><, if and pass.

@benknoble
Copy link
Collaborator

This makes me wonder about detecting (~> sep list-fuseable-values-steps collect) and list-fusing it, but that would really require knowledge of intermediate steps (like knowing that (>< user-func) is single-value output, maybe) to do.

@countvajhula
Copy link
Collaborator Author

Yeah, that's true. With something like #158 we should in principle be able to match certain special cases (where all functions are 1 × 1) to our existing deforestation pipeline. But the real challenge here in general is that user-func could return zero or more values rather than always one. I think when @dzoep first attempted to generalize our stream fusion implementation to multiple values, the performance dropped to be equivalent to naive Racket performance, so it wasn't viable for our deforestation of lists. But, given that values pipelines are likely much slower than lists, if we could handle this general case and have it perform as well as Racket lists, that would actually be a pretty great result. It could be a fallback in case better optimizations don't match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants