-
Notifications
You must be signed in to change notification settings - Fork 367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exception When Running AzureServiceBusSinkConnector With Premium Azure Service Bus #1446
Comments
Hello, @dallinb ! Thanks for bringing this one to our attention! I recently started looking into it, but after some time I've been unable to replicate it, so I'd like to ask some additional questions, if possible.
I'd appreciate the answers if you happen to have a spare moment 😉 |
Hi @GoMati-MU, Thanks for getting back. Casting my mind back:
In the meantime, we've developed our own connector, so not sure if I'd ever be able to get the time to recreate this problem. Under those circumstances, feel free to close this issue if it can't be reproduced. |
No worries at all, @dallinb ! All the thanks go to you for helping me out understand the issue :)
This may be the root cause of the problem for our connector, because most of the data sets I used for testing were pretty heavy in terms of amount of messages (up to 100K) but not that much in terms of size. Unfortunately I am pretty limited when it comes to resources (don't have access to Premium Service Bus) but I'll talk to the team to see if this can be set up. Premium API doesn't change in terms of API so I'd say this may be the valid road to take for debugging. I'll see whether I can do anything about this one for now, but I'm really happy you guys managed to resolve it for your usecase! Have a great one! |
Please try the latest release along with the extra kcql property |
Issue Guidelines
Please review these questions before submitting any issue?
What version of the Stream Reactor are you reporting this issue for?
8.1.11
Are you running the correct version of Kafka/Confluent for the Stream reactor release?
AFAIK
Do you have a supported version of the data source/sink .i.e Cassandra 3.0.9?
Using Kafka running in Docker Compose as the data source as this is a proof of concept/prototype. The error also doesn't happen when using the same rig but deploying to a Standard Azure Service Bus.
Have you read the docs?
Yes
What is the expected behaviour?
Messages to be transferred from Kafka to Azure Service Bus (happens when Service Bus is "Standard").
What was observed?
When the Azure Service Bus is "Premium" we get the following error status from Kafka Connect:
What is your Connect cluster configuration (connect-avro-distributed.properties)?
What is your connector properties configuration (my-connector.properties)?
Please provide full log files (redact and sensitive information)
Logs from Kafka Connect:
The text was updated successfully, but these errors were encountered: