Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support IGNORE|RESPECT NULLs clause in window functions #998

Merged
merged 14 commits into from
Oct 24, 2023
34 changes: 34 additions & 0 deletions src/ast/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -595,6 +595,14 @@ pub enum Expr {
/// `<search modifier>`
opt_search_modifier: Option<SearchModifier>,
},
RankFunction {
name: ObjectName,
expr: Box<Expr>,
offset: Option<Value>,
default: Option<Box<Expr>>,
respect_nulls: Option<bool>,
window: WindowSpec,
},
}

impl fmt::Display for Expr {
Expand Down Expand Up @@ -965,6 +973,32 @@ impl fmt::Display for Expr {

Ok(())
}
Expr::RankFunction {
name,
expr,
offset,
default,
respect_nulls,
window,
} => {
write!(f, "{name}({expr}")?;
if let Some(offset_value) = offset {
write!(f, ",{offset_value}")?;
}
if let Some(default_value) = default {
write!(f, ",{default_value}")?;
}
write!(f, ") ")?;
if let Some(respect_nulls_value) = respect_nulls {
if *respect_nulls_value {
write!(f, "RESPECT NULLS ")?;
} else {
write!(f, "IGNORE NULLS ")?;
}
}
write!(f, "OVER ({window})")?;
Ok(())
}
}
}
}
Expand Down
1 change: 1 addition & 0 deletions src/keywords.rs
Original file line number Diff line number Diff line change
Expand Up @@ -516,6 +516,7 @@ define_keywords!(
REPLACE,
REPLICATION,
RESET,
RESPECT,
RESTRICT,
RESULT,
RETAIN,
Expand Down
44 changes: 44 additions & 0 deletions src/parser/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -787,6 +787,9 @@ impl<'a> Parser<'a> {
}
Keyword::CASE => self.parse_case_expr(),
Keyword::CAST => self.parse_cast_expr(),
Keyword::LAG | Keyword::FIRST_VALUE | Keyword::LAST_VALUE | Keyword::LEAD if dialect_of!(self is SnowflakeDialect) => {
self.parse_rank_functions(ObjectName(vec![w.to_ident()]))
}
Keyword::TRY_CAST => self.parse_try_cast_expr(),
Keyword::SAFE_CAST => self.parse_safe_cast_expr(),
Keyword::EXISTS => self.parse_exists_expr(false),
Expand Down Expand Up @@ -7397,6 +7400,47 @@ impl<'a> Parser<'a> {
Ok(clauses)
}

pub fn parse_rank_functions(&mut self, name: ObjectName) -> Result<Expr, ParserError> {
self.expect_token(&Token::LParen)?;
let expr = self.parse_expr()?;
let offset = if self.consume_token(&Token::Comma) {
Some(self.parse_number_value()?)
} else {
None
};
let default = if self.consume_token(&Token::Comma) {
Some(self.parse_expr()?)
} else {
None
};
self.expect_token(&Token::RParen)?;

let respect_nulls;
if self.parse_keyword(Keyword::IGNORE) {
self.expect_keyword(Keyword::NULLS)?;
respect_nulls = Some(false)
} else if self.parse_keyword(Keyword::RESPECT) {
self.expect_keyword(Keyword::NULLS)?;
respect_nulls = Some(true)
} else {
respect_nulls = None
}

self.expect_keyword(Keyword::OVER)?;
self.expect_token(&Token::LParen)?;

let window = self.parse_window_spec()?;

Ok(Expr::RankFunction {
name,
expr: Box::new(expr),
offset,
default: default.map(Box::new),
respect_nulls,
window,
})
}

pub fn parse_merge(&mut self) -> Result<Statement, ParserError> {
let into = self.parse_keyword(Keyword::INTO);

Expand Down
37 changes: 37 additions & 0 deletions tests/sqlparser_snowflake.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1064,3 +1064,40 @@ fn test_snowflake_trim() {
snowflake().parse_sql_statements(error_sql).unwrap_err()
);
}

#[test]
fn test_snowflake_rank() {
let real_sql = "SELECT col_1, col_2, LAG(col_2) IGNORE NULLS OVER (ORDER BY col_1) FROM t1";
assert_eq!(snowflake().verified_stmt(real_sql).to_string(), real_sql);

let first_sql = "SELECT column1, column2, FIRST_VALUE(column2) OVER (PARTITION BY column1 ORDER BY column2 NULLS LAST) AS column2_first FROM t1";
assert_eq!(snowflake().verified_stmt(first_sql).to_string(), first_sql);

let sql_only_select = "SELECT LAG(col_2,1,0) OVER (PARTITION BY col_3 ORDER BY col_1)";
let select = snowflake().verified_only_select(sql_only_select);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this rather extends support for window function options Ignore/Respect NULLs vs snowflake specific, since the options are standard sql (e.g the test first_sql and sql_select_only should already be parsed without the MR)? If so it might be reasonable to instead extend the parse_function() to optionally accept an option where applicable like the following?

enum WindowFunctionOption {
	RespectNulls,
	IgnoreNulls
}

Since parse_function already contains most of the functionality being added, and prevents window functions from being parsed into potentially different types in the AST which might not be desirable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @iffyio !
I extracted the null clause to an enum, and moved it to parse_function.
I will appreciate any feedback.

Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nice!

One thing that stood out was the mismatch between the field name and the typename:

nulls_clause: Option<WindowFunctionOption>

was curious and took a quick look and it turns out the option is mostly called null treatment clause - maybe we can rename accordingly and so they're consistent?

null_treatment: Option<NullTreatment>

Other than that I think the changes look good!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @iffyio ! I looked for a better name but didn't found one, you did it!
Renamed :)

assert_eq!(
&Expr::RankFunction {
name: ObjectName(vec![Ident::new("LAG")]),
expr: Box::new(Expr::Identifier(Ident::new("col_2"))),
offset: Some(Value::Number("1".parse().unwrap(), false)),
default: Some(Box::new(Expr::Value(number("0")))),
respect_nulls: None,
window: WindowSpec {
partition_by: vec![Expr::Identifier(Ident {
value: "col_3".to_string(),
quote_style: None,
})],
order_by: vec![OrderByExpr {
expr: Expr::Identifier(Ident {
value: "col_1".to_string(),
quote_style: None,
}),
asc: None,
nulls_first: None,
}],
window_frame: None
}
},
expr_from_projection(only(&select.projection))
);
}
Loading