Document when zio-kafka is faster than raw java kafka

[erikvanoosten]

No description provided.

[svroonland]

when processing needs more than 1.2 ms per 1k records, a zio-kafka based program will have higher throughput

I'm not sure we can conclude that. The zio-kafka part will still be 1.2 ms slower per 1k records than fetching it via the plain kafka client, and on top of that you get the extra processing time which will slow the system down. I don't see how the processing time cancels the overhead.

[erikvanoosten]

In a week or so I'll have time to write a longer article that will hopefully explain my intuition on this 😄

[svroonland]

Be sure to take the new throughput results into account in your new calculation :)

[svroonland]

In ZioKafkaConsumerBenchmark.throughput:

Consumer
             .plainStream(Subscription.topics(topic1), Serde.byteArray, Serde.byteArray)
             .tap { _ =>
               counter.updateAndGet(_ + 1).flatMap(count => Consumer.stopConsumption.when(count == recordCount))
             }

Do you think it would be more fair to replace the tap with a tapChunk? In the manual kafka equivalent benchmark, we also process records by the batch instead of per record.

[erikvanoosten]

Do you think it would be more fair to replace the tap with a tapChunk?

Yes, I agree. Will you make a PR?

[svroonland]

See #1409