python写入kafka数据时报错解决方案

报错如下:

Traceback (most recent call last):
  File "/mnt/disk1/cdp/icu/icu_reades2kafka_v4_bigdata_v4_shuashu.py", line 23, in send_topic_msg
    record_metadata = result.get(timeout=10)
  File "/home/airflow/anaconda3/envs/airflow_env/lib/python3.7/site-packages/kafka/producer/future.py", line 65, in get
    raise self.exception # pylint: disable-msg=raising-bad-type
kafka.errors.MessageSizeTooLargeError: [Error 10] MessageSizeTooLargeError

解决方案:

如果有kafka manager的话,

可以在kafka manager上修改单个topic的配置

修改这个配置即可解决:max.message.bytes = 10485760

你可能感兴趣的:(kafka,python)