Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep PipelineChecker in the end of all PhysicalOptimizer rules #5076

Closed
Fischer0522 opened this issue Nov 30, 2024 · 2 comments · Fixed by #5092
Closed

Keep PipelineChecker in the end of all PhysicalOptimizer rules #5076

Fischer0522 opened this issue Nov 30, 2024 · 2 comments · Fixed by #5092
Labels
C-enhancement Category Enhancements

Comments

@Fischer0522
Copy link

What type of enhancement is this?

Refactor

What does the enhancement do?

Currently, datafusion places PipelineChecker at the end of the PhysicalOptimizer rules and uses it as a final gatekeeping rule. Therefore, all custom requests should be added before it to ensure that the plans after custom optimization (like WindowedSort or RemoveDuplicate) can still meet the checks of PipelineChecker.

In datafusion, PipelineChecker is described as follow:

            // The PipelineChecker rule will reject non-runnable query plans that use
            // pipeline-breaking operators on infinite input(s). The rule generates a
            // diagnostic error message when this happens. It makes no changes to the
            // given query plan; i.e. it only acts as a final gatekeeping rule.
            Arc::new(PipelineChecker::new()),

Implementation challenges

No response

@Fischer0522 Fischer0522 added the C-enhancement Category Enhancements label Nov 30, 2024
@killme2008
Copy link
Contributor

@Fischer0522 Thanks for your suggestion.

@waynexia What do you think?

@waynexia
Copy link
Member

waynexia commented Dec 4, 2024

Thanks for pointing this out @Fischer0522 ❤️ The patch is in #5092

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category Enhancements
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants