-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(ingest/fivetran): Add way to not add schema to the destination/source urn. #12314
base: master
Are you sure you want to change the base?
fix(ingest/fivetran): Add way to not add schema to the destination/source urn. #12314
Conversation
Kafka connector adds connector name to schema which should not be part of the urn.
Codecov ReportAttention: Patch coverage is
... and 201 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
include_schema: bool = pydantic.Field( | ||
default=True, | ||
description="Include schema in the dataset URN. In some cases, the schema is not relevant to the dataset URN and Fivetran sets it to the connector name.", | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
include_schema
might be too generic and could lead to misunderstandings. WDYT about something like include_schema_in_urn
instead?
I'm also a bit confused by the description: 'Include schema in the dataset URN' followed by 'Fivetran sets it to the connector name.' Is it referring to the dataset or the connector?
metadata-ingestion/src/datahub/ingestion/source/fivetran/config.py
Outdated
Show resolved
Hide resolved
…g.py Co-authored-by: Sergio Gómez Villamor <[email protected]>
Kafka connector/Confluent cloud connector adds the connector name to the schema, which should not be part of the urn.
Checklist