Convert MySQL datetime string to ISO8601 #122
Conversation
notfelineit
changed the title
COPY COMPLETED
| @@ -160,6 +161,8 @@ func parseValue(val sqltypes.Value, columnType string) sqltypes.Value { | |||
| } else if strings.HasPrefix(columnType, "set") { | |||
| values := parseEnumOrSetValues(columnType) | |||
| return mapSetValue(val, values) | |||
There was a problem hiding this comment.
All of the enum and set code above is wrong after vitessio/vitess#15723
That work is in v20+ in OSS and private.
There was a problem hiding this comment.
I think I'll remove it once everyone is on 20+ 😄 By "wrong" do you mean "unnecessary" or do you mean it will actually return the wrong values?
There was a problem hiding this comment.
It would be wrong if the user has string values in the ENUM/SET that can be converted to integers. For example, the string value may be 2 but it's not the index value of 2.
And we can do this work only when necessary by doing it conditional on:
- We're NOT in the copy phase
- The
EnumSetStringValues!= TRUE
See: https://github.com/vitessio/vitess/blob/main/changelog/20.0/20.0.0/summary.md#enum-set-vstream
That is exactly what they did in the Debezium connector.
There was a problem hiding this comment.
👍 We will address this in the next PR!
cmd/internal/types.go
Outdated
| @@ -160,6 +161,8 @@ func parseValue(val sqltypes.Value, columnType string) sqltypes.Value { | |||
| } else if strings.HasPrefix(columnType, "set") { | |||
| values := parseEnumOrSetValues(columnType) | |||
| return mapSetValue(val, values) | |||
| } else if lowerCased := strings.ToLower(columnType); strings.HasPrefix(lowerCased, "time") || strings.HasPrefix(lowerCased, "date") { | |||
There was a problem hiding this comment.
Do we not have an actual type here? If not, then using the MySQL column type string that we include works.
There was a problem hiding this comment.
Do you mean these methods? I can use them but wondering how to handle date?
There was a problem hiding this comment.
note: we talked offline and I modified the code to use Type!
Airbyte specifies that MySQL datetime/timestamp/date strings need to be parsed into ISO8601 format. This PR does that and also adds an extra log about whether we are looking for a COPY COMPLETED event or a stop position.