-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize isnan
(2-5x faster)
#12889
Optimize isnan
(2-5x faster)
#12889
Conversation
.collect::<$RETURN_TYPE>() | ||
}}; | ||
} | ||
use datafusion_common::Result; | ||
|
||
/// Isnan SQL function | ||
pub fn isnan(args: &[ArrayRef]) -> Result<ArrayRef> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this function is unused, and the real one is in datafusion/functions/src/math/nans.rs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are absolutely right.
Changing the right function does indeed give a speed-up.
I have left out the cleanup of the isnan
of math_expressions.rs out of this PR now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR #12917 to remove math_expressions.rs
3978dc4
to
62fc31e
Compare
isnan
using BooleanArray::from_unaryisnan
(2-5x faster)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @simonvandel
Thanks @simonvandel and @jonahgao -- I merged this PR up to resolve a conflict |
Which issue does this PR close?
Closes #.
Rationale for this change
Apply same idea from #12881 to
isnan
.What changes are included in this PR?
Are these changes tested?
Yes, with existing tests.
Are there any user-facing changes?
Yes, faster
is_nan
function.