15 kafka监控 一篇就够

wget https://raw.githubusercontent.com/prometheus/jmx_exporter/master/example_configs/kafka-2_0_0.yml

wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar

15.1解压安装

mkdir kafka_exporter

chown -R prometheus2:prometheus2 /home/monitor/kafka_exporter

文件放入kafka_exporter

15.2修改kafka启动文件

cp kafka-server-start.sh kafka-server-start-bak.sh

vim kafka-server-start.sh

kafka monitor

export JMX_PORT="9999"

export KAFKA_OPTS="-javaagent:/home/monitor/kafka_exporter/jmx_prometheus_javaagent-0.12.0.jar=9991:/home/monitor/kafka_exporter/kafka-2_0_0.yml"

image.png

bin/kafka-server-stop.sh

nohup bin/kafka-server-start.sh config/server.properties &

15.3添加到Prometheus

• job_name: '467kafkaCluster'

  static_configs:

• targets: ['172.16.10.4:9991']

• targets: ['172.16.10.6:9991']

• targets: ['172.16.10.7:9991']

15.4 添加grafana

Import dashboard 10466 10973选择数据源 Prometheus

15.5 监控指标说明

broker度量指标

正在复制的分区数量 - 关键指标under replicated partitions

kafka_cluster_partition_underreplicated 各个topic patition

kafka_server_replicamanager_underreplicatedpartitions 总

活跃控制器数量

请求处理器空闲率

请求度量指标

topic度量指标

主题流入字节速率

主题流出字节速率

主题流入消息速率

partition度量指标

分区数量

首领分区数量

离线分区数量

分区占用磁盘大小

分区文件数量

分区消息最大偏移量

分区消息最小偏移量