Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the behavior of the standard load methods in targets #2137

Open
pnadolny13 opened this issue Jan 8, 2024 · 0 comments
Open

Improve the behavior of the standard load methods in targets #2137

pnadolny13 opened this issue Jan 8, 2024 · 0 comments

Comments

@pnadolny13
Copy link
Contributor

Based on a discussion in target-postgres MeltanoLabs/target-postgres#266.

The current Singer convention in targets is to upsert records. If key properties are set then the target will update any records that exist instead of inserting.

Thoughts are it relates to the standard load_method configs:

  • Should the default actually be upsert? Or do we want to remove the default all together to make this config become more of an override to force the target to act a certain way vs a required value?
  • We should implement logic so that the target throws an error if someone selects upsert but theres no key properties available
  • If the default behavior of the SDK was always to upsert then we should change the logic to first check the load_method, if its set to append-only then dont upsert even if key properties are available.
  • Do existing targets get this functionality out of the box properly or is there anything they need to implement? If so, we should document it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant