From 33d97e85c8241c1c67146cb3e19aff0bb00becde Mon Sep 17 00:00:00 2001 From: Nghia Truong <7416935+ttnghia@users.noreply.github.com> Date: Sat, 14 Dec 2024 13:09:31 -0800 Subject: [PATCH] Support `trunc` and `date_trunc` SQL functions (#11833) * Implement GpuOverride Signed-off-by: Nghia Truong * Implement `GpuTruncDateTime` Signed-off-by: Nghia Truong * Do not fallback Signed-off-by: Nghia Truong * All variant input will be converted to just one Signed-off-by: Nghia Truong * Add generated docs Signed-off-by: Nghia Truong * Add tests Signed-off-by: Nghia Truong * Fix parameter types Signed-off-by: Nghia Truong * Fix test Signed-off-by: Nghia Truong * Change abstract class to trait Signed-off-by: Nghia Truong * Fix expression order and implement `TimeZoneAwareExpression` Signed-off-by: Nghia Truong * Update generated docs Signed-off-by: Nghia Truong * Update generated docs Signed-off-by: Nghia Truong * Add generated docs Signed-off-by: Nghia Truong * Add generated docs Signed-off-by: Nghia Truong * Allow non-utc timezone for timestamp tests Signed-off-by: Nghia Truong * Adopt to JNI changes Signed-off-by: Nghia Truong * Rewrite all classes Signed-off-by: Nghia Truong * Rename variable Signed-off-by: Nghia Truong --------- Signed-off-by: Nghia Truong --- .../advanced_configs.md | 2 + docs/supported_ops.md | 632 +++++++++++------- .../src/main/python/date_time_test.py | 75 ++- .../nvidia/spark/rapids/GpuOverrides.scala | 14 + .../sql/rapids/datetimeExpressions.scala | 157 ++++- tools/generated_files/320/operatorsScore.csv | 2 + tools/generated_files/320/supportedExprs.csv | 6 + tools/generated_files/321/operatorsScore.csv | 2 + tools/generated_files/321/supportedExprs.csv | 6 + tools/generated_files/330/operatorsScore.csv | 2 + tools/generated_files/330/supportedExprs.csv | 6 + tools/generated_files/331/operatorsScore.csv | 2 + tools/generated_files/331/supportedExprs.csv | 6 + tools/generated_files/340/operatorsScore.csv | 2 + tools/generated_files/340/supportedExprs.csv | 6 + tools/generated_files/342/operatorsScore.csv | 2 + tools/generated_files/342/supportedExprs.csv | 6 + tools/generated_files/400/operatorsScore.csv | 2 + tools/generated_files/400/supportedExprs.csv | 6 + tools/generated_files/operatorsScore.csv | 2 + tools/generated_files/supportedExprs.csv | 6 + 21 files changed, 702 insertions(+), 242 deletions(-) diff --git a/docs/additional-functionality/advanced_configs.md b/docs/additional-functionality/advanced_configs.md index a4427d9495a..5519e56b419 100644 --- a/docs/additional-functionality/advanced_configs.md +++ b/docs/additional-functionality/advanced_configs.md @@ -386,6 +386,8 @@ Name | SQL Function(s) | Description | Default Value | Notes spark.rapids.sql.expression.ToUnixTimestamp|`to_unix_timestamp`|Returns the UNIX timestamp of the given time|true|None| spark.rapids.sql.expression.TransformKeys|`transform_keys`|Transform keys in a map using a transform function|true|None| spark.rapids.sql.expression.TransformValues|`transform_values`|Transform values in a map using a transform function|true|None| +spark.rapids.sql.expression.TruncDate|`trunc`|Truncate the date to the unit specified by the given string format|true|None| +spark.rapids.sql.expression.TruncTimestamp|`date_trunc`|Truncate the timestamp to the unit specified by the given string format|true|None| spark.rapids.sql.expression.UnaryMinus|`negative`|Negate a numeric value|true|None| spark.rapids.sql.expression.UnaryPositive|`positive`|A numeric value with a + in front of it|true|None| spark.rapids.sql.expression.UnboundedFollowing$| |Special boundary for a window frame, indicating all rows preceding the current row|true|None| diff --git a/docs/supported_ops.md b/docs/supported_ops.md index be52afd9799..2ac5a27daa7 100644 --- a/docs/supported_ops.md +++ b/docs/supported_ops.md @@ -17702,6 +17702,154 @@ are limited. YEARMONTH +TruncDate +`trunc` +Truncate the date to the unit specified by the given string format +None +project +date + + + + + + + +S + + + + + + + + + + + + + + +format + + + + + + + + + +S + + + + + + + + + + + + +result + + + + + + + +S + + + + + + + + + + + + + + +TruncTimestamp +`date_trunc` +Truncate the timestamp to the unit specified by the given string format +None +project +format + + + + + + + + + +S + + + + + + + + + + + + +date + + + + + + + + +PS
UTC is only supported TZ for TIMESTAMP + + + + + + + + + + + + + +result + + + + + + + + +PS
UTC is only supported TZ for TIMESTAMP + + + + + + + + + + + + + UnaryMinus `negative` Negate a numeric value @@ -17926,6 +18074,34 @@ are limited. +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + UnboundedPreceding$ Special boundary for a window frame, indicating all rows preceding the current row @@ -18079,34 +18255,6 @@ are limited. -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - Upper `ucase`, `upper` String uppercase operator @@ -18357,6 +18505,34 @@ are limited. NS +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + XxHash64 `xxhash64` xxhash64 hash operator @@ -18673,34 +18849,6 @@ are limited. S -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - ApproximatePercentile `approx_percentile`, `percentile_approx` Approximate percentile @@ -18891,6 +19039,34 @@ are limited. +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + Average `avg`, `mean` Average aggregate operator @@ -19181,34 +19357,6 @@ are limited. -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - CollectSet `collect_set` Collect a set of unique elements, not supported in reduction @@ -19354,6 +19502,34 @@ are limited. +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + Count `count` Count aggregate operator @@ -19638,38 +19814,10 @@ are limited. NS PS
UTC is only supported TZ for child TIMESTAMP;
unsupported child types CALENDAR, UDT, DAYTIME, YEARMONTH PS
UTC is only supported TZ for child TIMESTAMP;
unsupported child types CALENDAR, UDT, DAYTIME, YEARMONTH -PS
UTC is only supported TZ for child TIMESTAMP;
unsupported child types CALENDAR, UDT, DAYTIME, YEARMONTH -NS -NS -NS - - -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH +PS
UTC is only supported TZ for child TIMESTAMP;
unsupported child types CALENDAR, UDT, DAYTIME, YEARMONTH +NS +NS +NS Last @@ -19817,6 +19965,34 @@ are limited. NS +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + Max `max` Max aggregate operator @@ -20106,34 +20282,6 @@ are limited. NS -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - Min `min` Min aggregate operator @@ -20279,6 +20427,34 @@ are limited. +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + MinBy `min_by` MinBy aggregate operator. It may produce different results than CPU when multiple rows in a group have same minimum value in the ordering column and different associated values in the value column. @@ -20613,34 +20789,6 @@ are limited. -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - PivotFirst PivotFirst operator @@ -20785,6 +20933,34 @@ are limited. NS +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + StddevPop `stddev_pop` Aggregation computing population standard deviation @@ -21075,34 +21251,6 @@ are limited. -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - Sum `sum` Sum aggregate operator @@ -21248,6 +21396,34 @@ are limited. +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + VariancePop `var_pop` Aggregation computing population variance @@ -21538,34 +21714,6 @@ are limited. -Expression -SQL Functions(s) -Description -Notes -Context -Param/Output -BOOLEAN -BYTE -SHORT -INT -LONG -FLOAT -DOUBLE -DATE -TIMESTAMP -STRING -DECIMAL -NULL -BINARY -CALENDAR -ARRAY -MAP -STRUCT -UDT -DAYTIME -YEARMONTH - - NormalizeNaNAndZero Normalize NaN and zero @@ -21645,6 +21793,34 @@ are limited. NS +Expression +SQL Functions(s) +Description +Notes +Context +Param/Output +BOOLEAN +BYTE +SHORT +INT +LONG +FLOAT +DOUBLE +DATE +TIMESTAMP +STRING +DECIMAL +NULL +BINARY +CALENDAR +ARRAY +MAP +STRUCT +UDT +DAYTIME +YEARMONTH + + HiveGenericUDF Hive Generic UDF, the UDF can choose to implement a RAPIDS accelerated interface to get better performance diff --git a/integration_tests/src/main/python/date_time_test.py b/integration_tests/src/main/python/date_time_test.py index 1a7024dac85..d29e6a5b5fa 100644 --- a/integration_tests/src/main/python/date_time_test.py +++ b/integration_tests/src/main/python/date_time_test.py @@ -321,7 +321,7 @@ def test_unsupported_fallback_to_unix_timestamp(data_gen): spark, [("a", data_gen), ("b", string_gen)], length=10).selectExpr( "to_unix_timestamp(a, b)"), "ToUnixTimestamp") - + supported_timezones = ["Asia/Shanghai", "UTC", "UTC+0", "UTC-0", "GMT", "GMT+0", "GMT-0", "EST", "MST", "VST"] unsupported_timezones = ["PST", "NST", "AST", "America/Los_Angeles", "America/New_York", "America/Chicago"] @@ -681,7 +681,7 @@ def test_unsupported_fallback_to_date(): conf) -# (-62135510400, 253402214400) is the range of seconds that can be represented by timestamp_seconds +# (-62135510400, 253402214400) is the range of seconds that can be represented by timestamp_seconds # considering the influence of time zone. ts_float_gen = SetValuesGen(FloatType(), [0.0, -0.0, 1.0, -1.0, 1.234567, -1.234567, 16777215.0, float('inf'), float('-inf'), float('nan')]) seconds_gens = [LongGen(min_val=-62135510400, max_val=253402214400), IntegerGen(), ShortGen(), ByteGen(), @@ -710,7 +710,7 @@ def test_timestamp_seconds_rounding_necessary(data_gen): lambda spark : unary_op_df(spark, data_gen).selectExpr("timestamp_seconds(a)").collect(), conf={}, error_message='Rounding necessary') - + @pytest.mark.parametrize('data_gen', [DecimalGen(19, 6), DecimalGen(20, 6)], ids=idfn) @allow_non_gpu(*non_utc_allow) def test_timestamp_seconds_decimal_overflow(data_gen): @@ -725,7 +725,7 @@ def test_timestamp_seconds_decimal_overflow(data_gen): def test_timestamp_millis(data_gen): assert_gpu_and_cpu_are_equal_collect( lambda spark : unary_op_df(spark, data_gen).selectExpr("timestamp_millis(a)")) - + @allow_non_gpu(*non_utc_allow) def test_timestamp_millis_long_overflow(): assert_gpu_and_cpu_error( @@ -751,3 +751,70 @@ def test_date_to_timestamp(parser_policy): conf = { "spark.sql.legacy.timeParserPolicy": parser_policy, "spark.rapids.sql.incompatibleDateFormats.enabled": True}) + +# Generate format strings, which are case insensitive and have some garbage rows. +trunc_date_format_gen = StringGen('(?i:YEAR|YYYY|YY|QUARTER|MONTH|MM|MON|WEEK)') \ + .with_special_pattern('invalid', weight=50) +trunc_timestamp_format_gen = StringGen('(?i:YEAR|YYYY|YY|QUARTER|MONTH|MM|MON|WEEK|DAY|DD|HOUR|MINUTE|SECOND|MILLISECOND|MICROSECOND)') \ + .with_special_pattern('invalid', weight=50) + +@pytest.mark.parametrize('data_gen', [date_gen], ids=idfn) +@pytest.mark.parametrize('format_gen', [trunc_date_format_gen], ids=idfn) +def test_trunc_date_full_input(data_gen, format_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : two_col_df(spark, data_gen, format_gen).selectExpr('trunc(a, b)')) + +@allow_non_gpu(*non_utc_tz_allow) +@pytest.mark.parametrize('format_gen', [trunc_timestamp_format_gen], ids=idfn) +@pytest.mark.parametrize('data_gen', [timestamp_gen], ids=idfn) +def test_trunc_timestamp_full_input(format_gen, data_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : two_col_df(spark, format_gen, data_gen).selectExpr('date_trunc(a, b)')) + +@pytest.mark.parametrize('format_gen', [trunc_date_format_gen], ids=idfn) +def test_trunc_date_single_value(format_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : unary_op_df(spark, format_gen).selectExpr('trunc("1980-05-18", a)')) + +@allow_non_gpu(*non_utc_tz_allow) +@pytest.mark.parametrize('format_gen', [trunc_timestamp_format_gen], ids=idfn) +def test_trunc_timestamp_single_value(format_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : unary_op_df(spark, format_gen).selectExpr( + 'date_trunc(a, "1980-05-18T09:32:05.359")')) + +@pytest.mark.parametrize('data_gen', [date_gen], ids=idfn) +def test_trunc_date_single_format(data_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : unary_op_df(spark, data_gen).selectExpr( + 'trunc(a, "YEAR")', + 'trunc(a, "YYYY")', + 'trunc(a, "YY")', + 'trunc(a, "QUARTER")', + 'trunc(a, "MONTH")', + 'trunc(a, "MM")', + 'trunc(a, "MON")', + 'trunc(a, "WEEK")', + 'trunc(a, "invalid")')) + +@allow_non_gpu(*non_utc_tz_allow) +@pytest.mark.parametrize('data_gen', [timestamp_gen], ids=idfn) +def test_trunc_timestamp_single_format(data_gen): + assert_gpu_and_cpu_are_equal_collect( + lambda spark : unary_op_df(spark, data_gen).selectExpr( + 'date_trunc("YEAR", a)', + 'date_trunc("YYYY", a)', + 'date_trunc("YY", a)', + 'date_trunc("QUARTER", a)', + 'date_trunc("MONTH", a)', + 'date_trunc("MM", a)', + 'date_trunc("MON", a)', + 'date_trunc("WEEK", a)', + 'date_trunc("DAY", a)', + 'date_trunc("DD", a)', + 'date_trunc("HOUR", a)', + 'date_trunc("MINUTE", a)', + 'date_trunc("SECOND", a)', + 'date_trunc("MILLISECOND", a)', + 'date_trunc("MICROSECOND", a)', + 'date_trunc("invalid", a)')) diff --git a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala b/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala index bd9aad97772..12ebfc01862 100644 --- a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala +++ b/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala @@ -1822,6 +1822,20 @@ object GpuOverrides extends Logging { ParamCheck("round", TypeSig.lit(TypeEnum.BOOLEAN), TypeSig.BOOLEAN))), (a, conf, p, r) => new MonthsBetweenExprMeta(a, conf, p, r) ), + expr[TruncDate]( + "Truncate the date to the unit specified by the given string format", + ExprChecks.binaryProject(TypeSig.DATE, TypeSig.DATE, + ("date", TypeSig.DATE, TypeSig.DATE), + ("format", TypeSig.STRING, TypeSig.STRING)), + (a, conf, p, r) => new TruncDateExprMeta(a, conf, p, r) + ), + expr[TruncTimestamp]( + "Truncate the timestamp to the unit specified by the given string format", + ExprChecks.binaryProject(TypeSig.TIMESTAMP, TypeSig.TIMESTAMP, + ("format", TypeSig.STRING, TypeSig.STRING), + ("date", TypeSig.TIMESTAMP, TypeSig.TIMESTAMP)), + (a, conf, p, r) => new TruncTimestampExprMeta(a, conf, p, r) + ), expr[Pmod]( "Pmod", // Decimal support disabled https://github.com/NVIDIA/spark-rapids/issues/7553 diff --git a/sql-plugin/src/main/scala/org/apache/spark/sql/rapids/datetimeExpressions.scala b/sql-plugin/src/main/scala/org/apache/spark/sql/rapids/datetimeExpressions.scala index d08c598cba4..ad14ab400dd 100644 --- a/sql-plugin/src/main/scala/org/apache/spark/sql/rapids/datetimeExpressions.scala +++ b/sql-plugin/src/main/scala/org/apache/spark/sql/rapids/datetimeExpressions.scala @@ -27,10 +27,10 @@ import com.nvidia.spark.rapids.Arm._ import com.nvidia.spark.rapids.ExprMeta import com.nvidia.spark.rapids.GpuOverrides.{extractStringLit, getTimeParserPolicy} import com.nvidia.spark.rapids.RapidsPluginImplicits._ -import com.nvidia.spark.rapids.jni.GpuTimeZoneDB +import com.nvidia.spark.rapids.jni.{DateTimeUtils, GpuTimeZoneDB} import com.nvidia.spark.rapids.shims.{NullIntolerantShim, ShimBinaryExpression, ShimExpression} -import org.apache.spark.sql.catalyst.expressions.{BinaryExpression, ExpectsInputTypes, Expression, FromUnixTime, FromUTCTimestamp, ImplicitCastInputTypes, MonthsBetween, TimeZoneAwareExpression, ToUTCTimestamp} +import org.apache.spark.sql.catalyst.expressions.{BinaryExpression, ExpectsInputTypes, Expression, FromUnixTime, FromUTCTimestamp, ImplicitCastInputTypes, MonthsBetween, TimeZoneAwareExpression, ToUTCTimestamp, TruncDate, TruncTimestamp} import org.apache.spark.sql.catalyst.util.DateTimeConstants import org.apache.spark.sql.internal.SQLConf import org.apache.spark.sql.types._ @@ -417,12 +417,11 @@ abstract class UnixTimeExprMeta[A <: BinaryExpression with TimeZoneAwareExpressi } trait GpuNumberToTimestampUnaryExpression extends GpuUnaryExpression { - override def dataType: DataType = TimestampType override def outputTypeOverride: DType = DType.TIMESTAMP_MICROSECONDS /** - * Test whether if input * multiplier will cause Long-overflow. In Math.multiplyExact, + * Test whether if input * multiplier will cause Long-overflow. In Math.multiplyExact, * if there is an integer-overflow, then it will throw an ArithmeticException "long overflow" */ def checkLongMultiplicationOverflow(input: ColumnVector, multiplier: Long): Unit = { @@ -439,7 +438,7 @@ trait GpuNumberToTimestampUnaryExpression extends GpuUnaryExpression { } protected val convertTo : GpuColumnVector => ColumnVector - + override def doColumnar(input: GpuColumnVector): ColumnVector = { convertTo(input) } @@ -543,14 +542,14 @@ case class GpuSecondsToTimestamp(child: Expression) extends GpuNumberToTimestamp longs.asTimestampSeconds() } case _ => - throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + + throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + s"for SecondsToTimestamp ") } } case class GpuMillisToTimestamp(child: Expression) extends GpuNumberToTimestampUnaryExpression { protected lazy val convertTo: GpuColumnVector => ColumnVector = child.dataType match { - case LongType => + case LongType => (input: GpuColumnVector) => { checkLongMultiplicationOverflow(input.getBase, DateTimeConstants.MICROS_PER_MILLIS) input.getBase.asTimestampMilliseconds() @@ -563,7 +562,7 @@ case class GpuMillisToTimestamp(child: Expression) extends GpuNumberToTimestampU } } case _ => - throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + + throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + s"for MillisToTimestamp ") } } @@ -581,7 +580,7 @@ case class GpuMicrosToTimestamp(child: Expression) extends GpuNumberToTimestampU } } case _ => - throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + + throw new UnsupportedOperationException(s"Unsupport type ${child.dataType} " + s"for MicrosToTimestamp ") } } @@ -1110,7 +1109,7 @@ abstract class ConvertUTCTimestampExprMetaBase[INPUT <: BinaryExpression]( rule: DataFromReplacementRule) extends BinaryExprMeta[INPUT](expr, conf, parent, rule) { - protected[this] var timezoneId: ZoneId = null + protected[this] var timezoneId: ZoneId = null override def tagExprForGpu(): Unit = { extractStringLit(expr.right) match { @@ -1527,3 +1526,141 @@ case class GpuLastDay(startDate: Expression) override protected def doColumnar(input: GpuColumnVector): ColumnVector = input.getBase.lastDayOfMonth() } + +abstract class GpuTruncDateTime(fmtStr: Option[String]) extends GpuBinaryExpression + with ImplicitCastInputTypes with Serializable { + override def nullable: Boolean = true + + protected def truncate(datetimeCol: GpuColumnVector, fmtCol: GpuColumnVector): ColumnVector = { + DateTimeUtils.truncate(datetimeCol.getBase, fmtCol.getBase) + } + + protected def truncate(datetimeVal: GpuScalar, fmtCol: GpuColumnVector): ColumnVector = { + withResource(ColumnVector.fromScalar(datetimeVal.getBase, 1)) { datetimeCol => + DateTimeUtils.truncate(datetimeCol, fmtCol.getBase) + } + } + + protected def truncate(datetimeCol: GpuColumnVector, fmtVal: GpuScalar): ColumnVector = { + // fmtVal is unused, as it was extracted to `fmtStr` before. + fmtStr match { + case Some(fmt) => DateTimeUtils.truncate(datetimeCol.getBase, fmt) + case None => throw new IllegalArgumentException("Invalid format string.") + } + } + + protected def truncate(numRows: Int, datetimeVal: GpuScalar, fmtVal: GpuScalar): ColumnVector = { + // fmtVal is unused, as it was extracted to `fmtStr` before. + fmtStr match { + case Some(fmt) => + withResource(ColumnVector.fromScalar(datetimeVal.getBase, 1)) { datetimeCol => + val truncated = DateTimeUtils.truncate(datetimeCol, fmt) + if (numRows == 1) { + truncated + } else { + withResource(truncated) { _ => + withResource(truncated.getScalarElement(0)) { truncatedScalar => + ColumnVector.fromScalar(truncatedScalar, numRows) + } + } + } + } + case None => throw new IllegalArgumentException("Invalid format string.") + } + } +} + +case class GpuTruncDate(date: Expression, fmt: Expression, fmtStr: Option[String]) + extends GpuTruncDateTime(fmtStr) { + override def left: Expression = date + + override def right: Expression = fmt + + override def inputTypes: Seq[AbstractDataType] = Seq(DateType, StringType) + + override def dataType: DataType = DateType + + override def prettyName: String = "trunc" + + override def doColumnar(lhs: GpuColumnVector, rhs: GpuColumnVector): ColumnVector = { + truncate(lhs, rhs) + } + + override def doColumnar(lhs: GpuScalar, rhs: GpuColumnVector): ColumnVector = { + truncate(lhs, rhs) + } + + override def doColumnar(lhs: GpuColumnVector, rhs: GpuScalar): ColumnVector = { + truncate(lhs, rhs) + } + + override def doColumnar(numRows: Int, lhs: GpuScalar, rhs: GpuScalar): ColumnVector = { + truncate(numRows, lhs, rhs) + } +} + +case class GpuTruncTimestamp(fmt: Expression, timestamp: Expression, timeZoneId: Option[String], + fmtStr: Option[String]) + extends GpuTruncDateTime(fmtStr) with TimeZoneAwareExpression { + + override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression = { + copy(timeZoneId = Option(timeZoneId)) + } + + override def left: Expression = fmt + + override def right: Expression = timestamp + + override def inputTypes: Seq[AbstractDataType] = Seq(StringType, TimestampType) + + override def dataType: DataType = TimestampType + + override def prettyName: String = "date_trunc" + + // Since the input order of this class is opposite compared to the `GpuTruncDate` class, + // we need to switch `lhs` and `rhs` in the `doColumnar` methods below. + + override def doColumnar(lhs: GpuColumnVector, rhs: GpuColumnVector): ColumnVector = { + truncate(rhs, lhs) + } + + override def doColumnar(lhs: GpuScalar, rhs: GpuColumnVector): ColumnVector = { + truncate(rhs, lhs) + } + + override def doColumnar(lhs: GpuColumnVector, rhs: GpuScalar): ColumnVector = { + truncate(rhs, lhs) + } + + override def doColumnar(numRows: Int, lhs: GpuScalar, rhs: GpuScalar): ColumnVector = { + truncate(numRows, rhs, lhs) + } +} + +class TruncDateExprMeta(expr: TruncDate, + override val conf: RapidsConf, + override val parent: Option[RapidsMeta[_, _, _]], + rule: DataFromReplacementRule) + extends BinaryExprMeta[TruncDate](expr, conf, parent, rule) { + + // Store the format string as we need to process it on the CPU later on. + private val fmtStr = extractStringLit(expr.format) + + override def convertToGpu(date: Expression, format: Expression): GpuExpression = { + GpuTruncDate(date, format, fmtStr) + } +} + +class TruncTimestampExprMeta(expr: TruncTimestamp, + override val conf: RapidsConf, + override val parent: Option[RapidsMeta[_, _, _]], + rule: DataFromReplacementRule) + extends BinaryExprMeta[TruncTimestamp](expr, conf, parent, rule) { + + // Store the format string as we need to process it on the CPU later on. + private val fmtStr = extractStringLit(expr.format) + + override def convertToGpu(format: Expression, timestamp: Expression): GpuExpression = { + GpuTruncTimestamp(format, timestamp, expr.timeZoneId, fmtStr) + } +} diff --git a/tools/generated_files/320/operatorsScore.csv b/tools/generated_files/320/operatorsScore.csv index 19c999aa796..d8c4ca63adc 100644 --- a/tools/generated_files/320/operatorsScore.csv +++ b/tools/generated_files/320/operatorsScore.csv @@ -265,6 +265,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/320/supportedExprs.csv b/tools/generated_files/320/supportedExprs.csv index 573367d1d70..39d69c0836b 100644 --- a/tools/generated_files/320/supportedExprs.csv +++ b/tools/generated_files/320/supportedExprs.csv @@ -606,6 +606,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/321/operatorsScore.csv b/tools/generated_files/321/operatorsScore.csv index 19c999aa796..d8c4ca63adc 100644 --- a/tools/generated_files/321/operatorsScore.csv +++ b/tools/generated_files/321/operatorsScore.csv @@ -265,6 +265,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/321/supportedExprs.csv b/tools/generated_files/321/supportedExprs.csv index 573367d1d70..39d69c0836b 100644 --- a/tools/generated_files/321/supportedExprs.csv +++ b/tools/generated_files/321/supportedExprs.csv @@ -606,6 +606,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/330/operatorsScore.csv b/tools/generated_files/330/operatorsScore.csv index e5978fb9f1a..e86e30e606c 100644 --- a/tools/generated_files/330/operatorsScore.csv +++ b/tools/generated_files/330/operatorsScore.csv @@ -275,6 +275,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/330/supportedExprs.csv b/tools/generated_files/330/supportedExprs.csv index e1a4492676c..c8df29cfca5 100644 --- a/tools/generated_files/330/supportedExprs.csv +++ b/tools/generated_files/330/supportedExprs.csv @@ -627,6 +627,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/331/operatorsScore.csv b/tools/generated_files/331/operatorsScore.csv index b988344e702..229201ba885 100644 --- a/tools/generated_files/331/operatorsScore.csv +++ b/tools/generated_files/331/operatorsScore.csv @@ -276,6 +276,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/331/supportedExprs.csv b/tools/generated_files/331/supportedExprs.csv index 7329b2c4756..e217b6ce31d 100644 --- a/tools/generated_files/331/supportedExprs.csv +++ b/tools/generated_files/331/supportedExprs.csv @@ -629,6 +629,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/340/operatorsScore.csv b/tools/generated_files/340/operatorsScore.csv index b1e9198e58b..16ac93a02ba 100644 --- a/tools/generated_files/340/operatorsScore.csv +++ b/tools/generated_files/340/operatorsScore.csv @@ -277,6 +277,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/340/supportedExprs.csv b/tools/generated_files/340/supportedExprs.csv index 22b1f73d68b..0c2d6a74bc1 100644 --- a/tools/generated_files/340/supportedExprs.csv +++ b/tools/generated_files/340/supportedExprs.csv @@ -629,6 +629,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/342/operatorsScore.csv b/tools/generated_files/342/operatorsScore.csv index b1e9198e58b..16ac93a02ba 100644 --- a/tools/generated_files/342/operatorsScore.csv +++ b/tools/generated_files/342/operatorsScore.csv @@ -277,6 +277,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/342/supportedExprs.csv b/tools/generated_files/342/supportedExprs.csv index 22b1f73d68b..0c2d6a74bc1 100644 --- a/tools/generated_files/342/supportedExprs.csv +++ b/tools/generated_files/342/supportedExprs.csv @@ -629,6 +629,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/400/operatorsScore.csv b/tools/generated_files/400/operatorsScore.csv index 53791a06705..0a099fc2233 100644 --- a/tools/generated_files/400/operatorsScore.csv +++ b/tools/generated_files/400/operatorsScore.csv @@ -278,6 +278,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/400/supportedExprs.csv b/tools/generated_files/400/supportedExprs.csv index 0623b25bce0..d3f1f1b851a 100644 --- a/tools/generated_files/400/supportedExprs.csv +++ b/tools/generated_files/400/supportedExprs.csv @@ -635,6 +635,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,S,S UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS diff --git a/tools/generated_files/operatorsScore.csv b/tools/generated_files/operatorsScore.csv index 19c999aa796..d8c4ca63adc 100644 --- a/tools/generated_files/operatorsScore.csv +++ b/tools/generated_files/operatorsScore.csv @@ -265,6 +265,8 @@ ToUTCTimestamp,4 ToUnixTimestamp,4 TransformKeys,4 TransformValues,4 +TruncDate,4 +TruncTimestamp,4 UnaryMinus,4 UnaryPositive,4 UnboundedFollowing$,4 diff --git a/tools/generated_files/supportedExprs.csv b/tools/generated_files/supportedExprs.csv index 573367d1d70..39d69c0836b 100644 --- a/tools/generated_files/supportedExprs.csv +++ b/tools/generated_files/supportedExprs.csv @@ -606,6 +606,12 @@ TransformKeys,S,`transform_keys`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA, TransformValues,S,`transform_values`,None,project,argument,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA TransformValues,S,`transform_values`,None,project,function,S,S,S,S,S,S,S,S,PS,S,S,S,NS,NS,PS,PS,PS,NS,NS,NS TransformValues,S,`transform_values`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncDate,S,`trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,format,NA,NA,NA,NA,NA,NA,NA,NA,NA,S,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,date,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA +TruncTimestamp,S,`date_trunc`,None,project,result,NA,NA,NA,NA,NA,NA,NA,NA,PS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA UnaryMinus,S,`negative`,None,project,input,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,project,result,NA,S,S,S,S,S,S,NA,NA,NA,S,NA,NA,NS,NA,NA,NA,NA,NS,NS UnaryMinus,S,`negative`,None,AST,input,NA,NS,NS,S,S,S,S,NA,NA,NA,NS,NA,NA,NS,NA,NA,NA,NA,NS,NS