Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend expression fuzzer test to support decimal #9149

Closed
wants to merge 1 commit into from

Conversation

rui-mo
Copy link
Collaborator

@rui-mo rui-mo commented Mar 19, 2024

ArgumentTypeFuzzer could be used to generate argument types when constructing a
decimal expression in the fuzzer test. However, given a result type, it is
unable to produce argument types that satisfy the necessary constraints. To
address this limitation, argument type generators for Presto and Spark decimal
functions have been added. In this PR, ExpressionFuzzer takes a map from
function name to an instance of the argument generator. Custom generators
provided by Presto and Spark are used in the expression fuzzer test to generate
argument types. Experimental fuzzer tests with decimal type enabled will be
added in a follow-up PR.

#1968

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 19, 2024
Copy link

netlify bot commented Mar 19, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 70cf156
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/66d8f878d7dc8100087a45c3

@rui-mo
Copy link
Collaborator Author

rui-mo commented Mar 19, 2024

Hi @mbasmanova, I drafted this PR to support decimal in expression fuzzer test. Tested decimal_add and decimal_between locally for 10 minutes each. Below are some next-steps I can think of.

  • Verify other decimal functions as well (agg functions are tested in AggregationFuzzer, so they will not be covered in this PR).
  • Fix CI issues and enable velox_fuzzer_enable_decimal_type in CI.
  • Test decimal function registered as simple function when the relating PR is merged.

Besides that, do you have more items in mind which I should take care of to move forward? Thank you.

Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rui-mo Rui, thank you for extending Fuzzer to cover decimal types. Would you update the PR description to provide some details about the design of this extension? Are there any limitations?

@kagamiori @kgpai Krishna, Wei, would you help review this PR?

CC: @majetideepak @aditi-pandit

@rui-mo rui-mo force-pushed the wip_dec_fuzzer branch 4 times, most recently from 3714ccc to 3e340cd Compare March 26, 2024 10:00
@rui-mo rui-mo changed the title Support decimal in expression fuzzer test Extend expression fuzzer test to support decimal Mar 26, 2024
@rui-mo rui-mo marked this pull request as ready for review March 26, 2024 13:27
@rui-mo
Copy link
Collaborator Author

rui-mo commented Mar 26, 2024

This PR is updated and ready for review. Thanks.

@rui-mo rui-mo requested a review from mbasmanova March 26, 2024 13:30
@rui-mo rui-mo requested a review from mbasmanova March 28, 2024 07:38
@rui-mo rui-mo force-pushed the wip_dec_fuzzer branch 5 times, most recently from 6ee7848 to 70e0b27 Compare April 1, 2024 00:56
@rui-mo rui-mo marked this pull request as draft April 1, 2024 01:21
@rui-mo rui-mo force-pushed the wip_dec_fuzzer branch 3 times, most recently from 8251b61 to 569a1c8 Compare April 1, 2024 05:57
Copy link
Collaborator

@majetideepak majetideepak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workflow changes look good!

@rui-mo
Copy link
Collaborator Author

rui-mo commented Jul 18, 2024

@majetideepak @assignUser Thanks for your review. The workflow fails with below error as the main branch is used for testing. It should be fine after this PR is landed.

ERROR: unknown command line flag 'velox_fuzzer_enable_decimal_type'

@majetideepak
Copy link
Collaborator

@rui-mo A safer option is to move the dependent workflow changes to a new PR. It is always safer to avoid landing changes when CI is failing.

@rui-mo rui-mo force-pushed the wip_dec_fuzzer branch 2 times, most recently from ea7a940 to 325aef2 Compare July 19, 2024 01:57
@rui-mo
Copy link
Collaborator Author

rui-mo commented Jul 19, 2024

A safer option is to move the dependent workflow changes to a new PR. It is always safer to avoid landing changes when CI is failing.

@majetideepak Thanks for the suggestion. Removed workflow changes.

@majetideepak
Copy link
Collaborator

@rui-mo thanks! This PR is already approved and tagged for merging. Meta will merge this.

@assignUser assignUser added ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall and removed ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall labels Jul 19, 2024
@rui-mo
Copy link
Collaborator Author

rui-mo commented Aug 7, 2024

@xiaoxmeng Would you help import and merge this PR? Thanks.

@rui-mo
Copy link
Collaborator Author

rui-mo commented Aug 14, 2024

Hi @xiaoxmeng, does this PR also require some internal work from Meta? Wondering if there is any chance to land this change first. Thanks.

@facebook-github-bot
Copy link
Contributor

@gggrace14 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Collaborator

@assignUser assignUser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last commit in this PR was almost a month ago. Can we rebase it before merging please.

@assignUser assignUser removed the ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall label Sep 4, 2024
@rui-mo
Copy link
Collaborator Author

rui-mo commented Sep 5, 2024

@assignUser @gggrace14 Rebased this PR, and the macOS build failure looks relevant to #10886. Would you take a another look? Thank you.

Copy link
Collaborator

@assignUser assignUser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@assignUser assignUser added the ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall label Sep 5, 2024
@pedroerp
Copy link
Contributor

pedroerp commented Sep 6, 2024

We are seeing this problem where PRs that were created before we enabled the notification system do not trigger a ready-to-merge notification. Apologies for the delay on this one.

@facebook-github-bot
Copy link
Contributor

@gggrace14 merged this pull request in 030e439.

Copy link

Conbench analyzed the 1 benchmark run on commit 030e439f.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

@rui-mo
Copy link
Collaborator Author

rui-mo commented Sep 8, 2024

@gggrace14 @pedroerp Thanks for noticing and helping land this PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants