[QUESTION]:ilogtail输出到kafka,默认配置情况下出现消息过大被kafka拒绝 #1555
Answered
by
shalousun
hailangxing
asked this question in
Help
-
使用默认的MaxMessageBytes配置,会出现kafka Message was to large,service rejected it to avoid allocation error,从MaxMessageBytes上看是0.96m,没有超过kafka默认消息大小限制1m |
Beta Was this translation helpful? Give feedback.
Answered by
shalousun
May 14, 2024
Replies: 1 comment
-
@hailangxing 这种kafka的客户端都是根据服务端状态返回状态的错误了,直接把服务端和客户端的都提升就OK了。本质上一条日志采集上来,发送到kafka最终的message会附带一些信息会比原来的日志大。这个也不属于是iLogtail的问题,发送消息都会碰到。设置留足空余,极限值很容易被拒绝。通常建议是broker端的配置要比producer端高。 |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
messixukejia
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@hailangxing 这种kafka的客户端都是根据服务端状态返回状态的错误了,直接把服务端和客户端的都提升就OK了。本质上一条日志采集上来,发送到kafka最终的message会附带一些信息会比原来的日志大。这个也不属于是iLogtail的问题,发送消息都会碰到。设置留足空余,极限值很容易被拒绝。通常建议是broker端的配置要比producer端高。