You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Similar to #1781 we should add a default configuration enum to SQL targets to configure whether to run the target in insert (append only) mode or upsert mode. Some targets will want this behavior so they'll implement it in a custom way, its better to get in front of it so we have a consistent config option and enum for all targets.
In addition we should consider recommending insert as the default option even if upsert is support and a key property is provided. This avoids challenges with deduplicating within a sync (i.e. unsorted streams, no timestamp or incrementing sort key, etc.).
The text was updated successfully, but these errors were encountered:
There could also be the option to run in trunc and load mode. This mode would truncate the table prior to loading. Its a bit risky so we should document it well. Snowflake has the time travel feature that allows you to recover any dropped tables or truncated rows back to a certain timestamp (default 24hr retention) so its a little safer if the developer accidentally uses this feature and drops their data.
tayloramurphy
changed the title
SQL targets should allow configurable insert vs upsert behavior by default
feat: SQL targets should allow configurable insert vs upsert behavior by default
Jul 11, 2023
Similar to #1781 we should add a default configuration enum to SQL targets to configure whether to run the target in insert (append only) mode or upsert mode. Some targets will want this behavior so they'll implement it in a custom way, its better to get in front of it so we have a consistent config option and enum for all targets.
In addition we should consider recommending insert as the default option even if upsert is support and a key property is provided. This avoids challenges with deduplicating within a sync (i.e. unsorted streams, no timestamp or incrementing sort key, etc.).
The text was updated successfully, but these errors were encountered: