Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TaskPrefix assignment does not follow 'system.reuseExistingDestination' flag. #972

Open
jogrogan opened this issue Nov 16, 2023 · 0 comments
Labels

Comments

@jogrogan
Copy link
Collaborator

Subject of the issue

There is a corner case where two datastreams could share a task prefix & consumer group despite expecting to output to two different topics.

Steps to reproduce

Tell us how to reproduce this issue.
Create datastream name "stream-1".
Create datastream name "dedupe-1". (dedupe-1 will now contain taskPrefix 'stream-1')
Delete "stream-1"
Recreate "stream-1" with metadata "system.reuseExistingDestination=false".

Expected behaviour

I would expect both streams to have different taskPrefixes (maybe through some uuid if system.reuseExistingDestination is present), different destination topics, and different consumer groups.

Actual behaviour

Both datastreams will still have taskPrefix "stream-1" despite the fact that with system.reuseExistingDestination=false both streams now have completely different destination topics.
The GroupIdConstructor logic will then assign the same consumer group to both streams and thus only one will actually get data.

@jogrogan jogrogan added the bug label Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant