Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company. Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project, after Kubernetes.
Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.
部署 Prometheus我们使用 helm 来安装，首先安装 promethus 的仓库。
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
创建 Prometheus 命名空间。
kubectl create namespace prometheus
部署 Prometheus。
helm install prometheus prometheus-community/prometheus \
--namespace prometheus \
--set alertmanager.persistentVolume.storageClass="gp2" \
--set server.persistentVolume.storageClass="gp2"
输出的这个地址我们后面会用到。
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-server.prometheus.svc.cluster.local
确认 prometheus 命名空间中的所有 Pod 均处于 READY 状态。
kubectl get pods -n prometheus
输出如下。
NAME READY STATUS RESTARTS AGE
prometheus-alertmanager-58666cd457-x6n7r 2/2 Running 0 86s
prometheus-kube-state-metrics-5b8f9bdbbd-6hz7m 1/1 Running 0 86s
prometheus-node-exporter-2btvc 1/1 Running 0 87s
prometheus-node-exporter-2mps4 1/1 Running 0 87s
prometheus-node-exporter-lzjbg 1/1 Running 0 87s
prometheus-node-exporter-xxqcm 1/1 Running 0 87s
prometheus-pushgateway-55b7f76c47-rmkb7 1/1 Running 0 86s
prometheus-server-648894fdf8-zlgg4 2/2 Running 0 86s
为了访问 prometheus，我们做一下端口转发。
kubectl port-forward -n prometheus deploy/prometheus-server 8080:9090
在 Cloud9 环境里面，点击 Tools / Preview / Preview Running Application，在 URL 末端添加如下
/targets
可以看到如下信息。
grafana 也是使用 helm 安装，首先配置好 grafana 的仓库。
helm repo add grafana https://grafana.github.io/helm-charts
然后部署 grafana。
kubectl create namespace grafana
helm install grafana grafana/grafana \
--namespace grafana \
--set persistence.storageClassName="gp2" \
--set adminPassword='Wangzan@18' \
--set datasources."datasources\.yaml".apiVersion=1 \
--set datasources."datasources\.yaml".datasources[0].name=Prometheus \
--set datasources."datasources\.yaml".datasources[0].type=prometheus \
--set datasources."datasources\.yaml".datasources[0].url=http://prometheus-server.prometheus.svc.cluster.local \
--set datasources."datasources\.yaml".datasources[0].access=proxy \
--set datasources."datasources\.yaml".datasources[0].isDefault=true \
--set service.type=LoadBalancer
确定 grafana 是不是部署完成。
kubectl get all -n grafana
输出如下。
NAME READY STATUS RESTARTS AGE
pod/grafana-5d6dcc4b46-dl95q 1/1 Running 0 59s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana LoadBalancer 10.100.53.152 a5cf235fc45974b80bfbece27c104564-1893187232.eu-west-1.elb.amazonaws.com 80:32042/TCP 60s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/grafana 1/1 1 1 60s
NAME DESIRED CURRENT READY AGE
replicaset.apps/grafana-5d6dcc4b46 1 1 1 60s
直接打开 ELB 的地址即可访问，密码就是我们前面定义的，如果忘记了，可以通过下面命令获取。
kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
修改为 Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: grafana
name: grafana
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=600
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/conditions.grafana: >
[{"field":"host-header","hostHeaderConfig":{"values":["grafana.wzlinux.com"]}}]
alb.ingress.kubernetes.io/group.name: wzlinux
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-west-1:921283538843:certificate/e55a72ae-d9b5-4f77-bf6d-242691105231
alb.ingress.kubernetes.io/target-type: ip
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: grafana
port:
number: 80
点击+，选择import，然后再官网查找合适的模版编号。
有些模版可能过时了，有些参数数据显示不出来，需要自行调整 PromSQL。
