You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm looking to use a single Kafka producer for two slightly differnt use cases/message types. One that is just fire-and-forget and that will simply enqueue a message without wait for any acknowledgment.
For the second use case I'd like to make sure that the message was written to broker before returning a response - similar to what's described in #1174
It seems that the way to achieve this would be to set a low message timeout and wait for poll() to acknowledge message delivery via the callback.
Having a low global message timeout wouldn't be very good for the fire-and-forget use cause, so it seems that the best option would be to use the per-topic configuration.
Was there any progress on implementing this since #328 (comment) ? It looks like none of the APIs are exposed through the python client.
If there's still no way to provide per-topic config I'd be open to work on it and make a pull request, but would like to understand what the preferred approach is. maybe exposing the topic creation methods on the produce with a config dict would be enough?
Thanks in advance
Checklist
Please provide the following information:
confluent-kafka-python==2.20 librdkafka==2.2.0
Apache Kafka broker version: 3.4.0
Client configuration: {...}
Operating system:
Provide client logs (with 'debug': '..' as necessary)
Provide broker log excerpts
Critical issue
The text was updated successfully, but these errors were encountered:
swehner
changed the title
Different topic timeout configration
Different per-topic timeout configuration
Nov 13, 2023
Description
I'm looking to use a single Kafka producer for two slightly differnt use cases/message types. One that is just fire-and-forget and that will simply enqueue a message without wait for any acknowledgment.
For the second use case I'd like to make sure that the message was written to broker before returning a response - similar to what's described in #1174
It seems that the way to achieve this would be to set a low message timeout and wait for
poll()
to acknowledge message delivery via the callback.Having a low global message timeout wouldn't be very good for the fire-and-forget use cause, so it seems that the best option would be to use the per-topic configuration.
Was there any progress on implementing this since #328 (comment) ? It looks like none of the APIs are exposed through the python client.
If there's still no way to provide per-topic config I'd be open to work on it and make a pull request, but would like to understand what the preferred approach is. maybe exposing the topic creation methods on the produce with a config dict would be enough?
Thanks in advance
Checklist
Please provide the following information:
{...}
'debug': '..'
as necessary)The text was updated successfully, but these errors were encountered: