filebeat报错dropping too large message of size
filebeat报错:
dropping too large message of size 1714620.
原因:
kafka对每一条消息的大小进行了限制。
解决
kafka端
修改config/server.properties,添加以下配置
max_message_bytes=10000000
replica.fetch.max.bytes=10000000
修改后重启kafka
filebeat端
修改filebeat.yaml,添加以下配置
max_message_bytes: 10000000
output.kafka:
hosts: ["kafka1:9092","kafka2:9092","kafka3:9092"]
topic: 'topiclog'
max_message_bytes: 10000000
required_acks: -1
添加后重启filebeat