Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX : Incorrect NULL handling in BETWEEN expression #14007

Merged
merged 5 commits into from
Jan 5, 2025

Conversation

getChan
Copy link
Contributor

@getChan getChan commented Jan 4, 2025

Which issue does this PR close?

Closes #13976.

Rationale for this change

What changes are included in this PR?

in BETWEEN expression,
The type of the high value is determined by the type of the high value, not the type of the low value.

As a result, the coercion type in a BETWEEN expression is determined by comparing the low, high, and compare values.

let expr_type = expr.get_type(self.schema)?;
let low_type = low.get_type(self.schema)?;
let low_coerced_type = comparison_coercion(&expr_type, &low_type)
.ok_or_else(|| {
DataFusionError::Internal(format!(
"Failed to coerce types {expr_type} and {low_type} in BETWEEN expression"
))
})?;
let high_type = high.get_type(self.schema)?;
let high_coerced_type = comparison_coercion(&expr_type, &high_type)
.ok_or_else(|| {
DataFusionError::Internal(format!(
"Failed to coerce types {expr_type} and {high_type} in BETWEEN expression"
))
})?;
let coercion_type =
comparison_coercion(&low_coerced_type, &high_coerced_type)
.ok_or_else(|| {
DataFusionError::Internal(format!(
"Failed to coerce types {expr_type} and {high_type} in BETWEEN expression"
))
})?;

Are these changes tested?

unit tested, sqllogic tested.

Are there any user-facing changes?

in BETWEEN expression,
Type coercion will work as intended.

@github-actions github-actions bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Jan 4, 2025
@getChan getChan marked this pull request as draft January 4, 2025 09:46
@getChan getChan marked this pull request as ready for review January 4, 2025 10:19
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @getChan -- this is a nice fix ❤️

Thanks to @2010YOUY01 for filing the ticket.

@@ -426,7 +426,7 @@ impl TreeNodeRewriter for TypeCoercionRewriter<'_> {
))
})?;
let high_type = high.get_type(self.schema)?;
let high_coerced_type = comparison_coercion(&expr_type, &low_type)
let high_coerced_type = comparison_coercion(&expr_type, &high_type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 -- nice fix

@@ -0,0 +1,32 @@
# Licensed to the Apache Software Foundation (ASF) under one
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to move this file into this https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest/test_files/expr directory?

We can do it as a follow on PR too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. moved it.

@alamb
Copy link
Contributor

alamb commented Jan 5, 2025

Thanks again @getChan 🚀

@alamb alamb merged commit ecd9f36 into apache:main Jan 5, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect NULL handling in BETWEEN expression
2 participants