Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CDCRecordStream SchemaDeltas #1134

Closed
wants to merge 1 commit into from
Closed

Conversation

serprex
Copy link
Contributor

@serprex serprex commented Jan 23, 2024

Instead RelationRecord is passed in with rest of Records,
where connectors handle collecting schema deltas

Instead RelationRecord is passed in with rest of Records,
where connectors handle collecting schema deltas
@serprex
Copy link
Contributor Author

serprex commented Jan 23, 2024

Little silly passing these records down when we could append to a slice held in CDCRecordStream

@serprex
Copy link
Contributor Author

serprex commented Jan 24, 2024

Closing in favor of #1135

@serprex serprex closed this Jan 24, 2024
serprex added a commit that referenced this pull request Jan 24, 2024
Simpler alternative to #1134

Particularly convenient having exclusion info already transformed into maps/sets in pull connector

Idea is that `CDCRecordStream` is a synchronization object.
While `Records` is open everything is written to only by pull connector,
we only receive records out Records channel.
Pull connector relieves write access once `Records` closed,
allowing everything else to read out the rest.
In this regard `lastCheckpointSet` is unnecessary
& I eventually intend to get rid of it
@serprex serprex deleted the refactor-cdc-recordbatch branch February 5, 2024 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant