Skip to content

Commit

Permalink
Clickhouse: remove nullable mapping (#1344)
Browse files Browse the repository at this point in the history
It appears timeseries performance is a common use-case for Clickhouse,
wherein users often index based on a `DateTime` column. Clickhouse does
not allow indexing based on `Nullable` columns. Also having nullable
columns decreases performance.
Thus, this PR chooses to stay with the original mapping of DateTime and
Date (without the Nullable)

A decision is to be made in the future regarding whether we get user's
preference on setting a column as Nullable or not for Clickhouse mirrors
  • Loading branch information
Amogh-Bharadwaj authored Feb 21, 2024
1 parent 2c170e6 commit 127bca9
Showing 1 changed file with 0 additions and 3 deletions.
3 changes: 0 additions & 3 deletions flow/connectors/clickhouse/normalize.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,6 @@ func generateCreateTableSQLForNormalizedTable(
}
stmtBuilder.WriteString(fmt.Sprintf("`%s` DECIMAL(%d, %d), ",
colName, precision, scale))
case qvalue.QValueKindTimestamp, qvalue.QValueKindTimestampTZ, qvalue.QValueKindDate:
// 1st Jan 1970 is not a sane default which clickhouse sets, so we use nullable
stmtBuilder.WriteString(fmt.Sprintf("`%s` Nullable(%s), ", colName, clickhouseType))
default:
stmtBuilder.WriteString(fmt.Sprintf("`%s` %s, ", colName, clickhouseType))
}
Expand Down

0 comments on commit 127bca9

Please sign in to comment.