Does rdkafka-dotnet invoke kafka's commitSync or commitAsync?

[dbykat]

Does rdkafka-dotnet invoke kafka's commitSync or commitAsync functionality when committing consumer offsets (assuming enable.auto.commit=false) ?? If it defaults to synchronous commits, does the rdkafka-dotnet API expose the async version, and can you provide a brief code example? In our experience, we see Commit method returns a Task (which implies async), but our consumer appears to wait to consume the subsequent message until the prior commit is ack'd (which implies sync). Appreciate some clarification, as this mixed behavior has me a bit confused...

note: we are using this repo, not the Confluent version. We will likely switch over in the near future, but need to know this with regards to the Andreas Heider version for our current needs.

Thank you!

[ah-]

It's currently sync underneath. I think at the time there was only the sync API and I intended to switch it out for the async later and therefore gave it the Task-based API. But as it currently stands it gives you a task but then happens to block until it's done.

The confluent client probably implements async commit.

[dbykat]

Thanks for the lightning fast response! Sync makes sense considering the behavior we're seeing... what's odd is that each consumer commit task takes about half a second to complete, which is killing our throughput. Does this 500 millisecond latency sound like something you might expect (or might have experienced)?

We're also investigating the possibility of network issues or kafka config settings, but when we produce messages, the commits are propagated to all brokers extremely quickly. Only consumer commits have this half second latency. Any thoughts?

[ah-]

Yes 500 milliseconds sounds like it is in the right ballpark. Maybe you can get this down with a bit of tuning of broker/client settings, but I'm not sure it's worth the effort.

How often are you committing? Maybe just commit less often, then the latency would have way less impact.

The other option is to try out the confluent client, I just checked and that has proper async commits and they seem to be getting close to a release.