k8s学习-Kubernetes Ingress
1.1 什么是Ingress？
首先我们来思考用传统的web服务器，比如Nginx，如何处理这种场景？
比如使用Nginx充当一个反向代理服务器拦截外部请求，读取路由规则配置，转发相应的请求到后端服务。
kubernetes处理这种场景时，涉及到三个组件：
- 反向代理web服务器
负责拦截外部请求，比如Nginx、Apache、traefik等等。我一般以Deployment方式部署到kubernetes集群中，当然也可以用DeamonSet方式部署；这两种部署方式个人觉得有利有弊，感兴趣的请参考这篇文章。
- Ingress controller
k8s中的controller有很多，比如CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等等，大家最熟悉的应该是Deployment，它的作用就是监控集群的变化，使集群始终保持我们期望的最终状态(yml文件)。同理，Ingress controller的作用就是实时感知Ingress路由规则集合的变化，再与Api Server交互，获取Service、Pod在集群中的 IP等信息，然后发送给反向代理web服务器(比如Nginx)，刷新其路由配置信息，这就是它的服务发现机制。
- Ingress
Ingress 简单的理解就是原来需要改 Nginx 配置，然后配置各种域名对应哪个 Service，现在把这个动作抽象出来，变成一个 Ingress 对象，可以用 yaml 创建，每次不要去改 Nginx 了，直接改 yaml 然后创建/更新就行了。
1.2 创建Ingress 资源
Ingress资源时基于HTTP虚拟主机或URL的转发规则，需要强调的是，这是一条转发规则。它在资源配置清单中的spec字段中嵌套了rules、backend和tls等字段进行定义。
如下示例中定义了一个Ingress资源，其包含了一个转发规则：将发往myapp.test.com的请求，代理给一个名字为myapp的Service资源。
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.test.com
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
Ingress 中的spec字段是Ingress资源的核心组成部分，主要包含以下3个字段：
- rules：用于定义当前Ingress资源的转发规则列表；由rules定义规则，或没有匹配到规则时，所有的流量会转发到由backend定义的默认后端。
- backend：默认的后端用于服务那些没有匹配到任何规则的请求；定义Ingress资源时，必须要定义backend或rules两者之一，该字段用于让负载均衡器指定一个全局默认的后端。
- tls：TLS配置，目前仅支持通过默认端口443提供服务，如果要配置指定的列表成员指向不同的主机，则需要通过SNITLS扩展机制来支持该功能。
backend对象的定义由2个必要的字段组成：serviceName和servicePort，分别用于指定流量转发的后端目标Service资源名称和端口。
rules对象由一系列的配置的Ingress资源的host规则组成，这些host规则用于将一个主机上的某个URL映射到相关后端Service对象，其定义格式如下：
spec:
rules:
- hosts: >
http:
paths:
- path:
backend:
serviceName: >
servicePort: >
Ingress资源类型
Ingress的资源类型有以下4种：
1、单Service资源型Ingress
2、基于URL路径进行流量转发
3、基于主机名称的虚拟主机
4、TLS类型的Ingress资源
1.2 部署配置Ingress
1 .部署文件介绍、准备
默认下载最新的yaml：
wget https://github.com/kubernetes/ingress-nginx/blob/nginx-0.20.0/deploy/mandatory.yaml
准备镜像，从这里mandatory.yaml查看需要哪些镜像
| 镜像名称 | 版本 | 下载地址 |
|---|---|---|
| k8s.gcr.io/defaultbackend-amd64 | 1.5 | registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64 |
| kquay.io/kubernetes-ingress-controller/nginx-ingress-controller | 0.20.0 | registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller |
在每一个节点(Node)上下载镜像:
[root@k8s-node2 ~]# docker pull registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
1.5: Pulling from kubernetes_xingej/defaultbackend-amd64
26de8f6c1f4b: Pull complete
Digest: sha256:2cdff48ab9b20ca5f9b0ee48bf3c139c51d6fb1a15245966583bc371c121c238
Status: Downloaded newer image for registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
[root@k8s-node2 ~]# docker pull registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
0.20.0: Pulling from kubernetes_xingej/nginx-ingress-controller
8e41b996a802: Pull complete
f8e7d603ef88: Pull complete
610da4f3123f: Pull complete
62702a85e6c9: Pull complete
7cedf5b2083f: Pull complete
755f7fa719b6: Pull complete
6adbdd0c8aaf: Pull complete
2389d8b0d2a2: Pull complete
1ea794448393: Pull complete
bb0c388ee432: Pull complete
9626641c5a97: Pull complete
bd0bebb5ba38: Pull complete
Digest: sha256:3f06079f7727b2fb7ad5c97d8152eb622ae504674395dfa71fda7ce315aaaf30
Status: Downloaded newer image for registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
下载yaml文件并更新mandatory.yaml中的镜像地址（master上）
[root@k8s-master ~]# mkdir ingress-nginx
[root@k8s-master ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.20.0/deploy/mandatory.yaml
[root@k8s-master ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.20.0/deploy/provider/baremetal/service-nodeport.yaml #对外提供服务，如果不需要可以不下载
[root@k8s-master ingress-nginx]# sed -i 's#k8s.gcr.io/defaultbackend-amd64#registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64#g' mandatory.yaml #替换defaultbackend-amd64镜像地址
sed -i 's#quay.io/kubernetes-ingress-controller/nginx-ingress-controller#registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller#g' mandatory.yaml #替换nginx-ingress-controller镜像地址
[root@k8s-master ingress-nginx]# grep image mandatory.yaml #检查替换结果
# Any image is permissible as long as:
image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
如果无法下载可以手动下然后上传。地址
修改service-nodeport.yaml文件，添加NodePort端口，默认为随机端口
[root@k8s-master ingress-nginx]# cat service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 32080 #http
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 32443 #https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
部署nginx-ingress-controller
[root@k8s-master ingress-nginx]# kubectl apply -f mandatory.yaml
namespace/ingress-nginx created
deployment.apps/default-http-backend created
service/default-http-backend created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
[root@k8s-master ingress-nginx]# kubectl apply -f service-nodeport.yaml
service/ingress-nginx created
查看ingress-nginx组件状态
[root@k8s-master ingress-nginx]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.1.103.71 <none> 80/TCP 8m48s
ingress-nginx NodePort 10.1.243.42 <none> 80:32080/TCP,443:32443/TCP 6s
访问ingress-nginx服务，查看是否配置成功
http://192.168.200.130:32080/
default backend - 404
可以看到，提示404，这个因为当前ingress-nginx服务现在还没有后端服务，这是正常的
1.3 创建ingress-nginx后端服务
创建一个Service及后端Deployment(以nginx为例)
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: default
spec:
selector:
app: myapp
release: canary
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deploy
spec:
replicas: 5
selector:
matchLabels:
app: myapp
release: canary
template:
metadata:
labels:
app: myapp
release: canary
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v2
ports:
- name: httpd
containerPort: 80
创建相关服务及检查状态是否就绪
[root@k8s-master ingress-nginx]# kubectl get pods
NAME READY STATUS RESTARTS AGE
busvbox 1/1 Running 0 3h38m
busybox-7c84546778-9jzpx 1/1 Running 15 17d
busybox-7c84546778-rs5d9 1/1 Running 15 17d
httpd-595dc58589-c4hcv 1/1 Running 5 2d6h
httpd-595dc58589-lxj9w 1/1 Running 6 2d6h
httpd-595dc58589-v6wzw 1/1 Running 11 5d
kindly-robin-mychart-fb68878c8-g4cfx 1/1 Running 0 5h10m
myapp-deploy-77c5c86fdb-52bnt 1/1 Running 0 11m
myapp-deploy-77c5c86fdb-9xjdm 1/1 Running 0 11m
myapp-deploy-77c5c86fdb-kznvk 1/1 Running 0 11m
myapp-deploy-77c5c86fdb-sp22l 1/1 Running 0 11m
myapp-deploy-77c5c86fdb-zp2tj 1/1 Running 0 11m
将myapp添加至ingress-nginx中
apiVersion: apps/v1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.soft.com #生产中该域名应当可以被公网解析
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
将myapp添加至ingress-nginx中
vi ingress-myapp.yaml
apiVersion: apps/v1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.soft.com #生产中该域名应当可以被公网解析
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
[root@k8s-master ingress-nginx]# kubectl apply -f ingress-myapp.yaml ingress.extensions "ingress-myapp" created
配置域名解析，当前测试环境我们使用hosts文件进行解析，修改本机的host
C:\Windows\System32\drivers\etc
192.168.200.130 myapp.soft.com
浏览器访问域名
http://myapp.soft.com:32080/
Hello MyApp | Version: v2 | Pod Name
1.4 配置ingress https 证书访问
再创建一个Service及后端Deployment(以tomcat为例)
vi tomcat-deploy.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat
namespace: default
spec:
selector:
app: tomcat
release: canary
ports:
- name: http
port: 8080
targetPort: 8080
- name: ajp
port: 8009
targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
release: canary
template:
metadata:
labels:
app: tomcat
release: canary
spec:
containers:
- name: tomcat
image: tomcat:7-alpine
ports:
- name: httpd
containerPort: 8080
- name: ajp
containerPort: 8009
通过kubectl将其部署到K8S集群：
kubectl apply -f tomcat-deploy.yaml```
```bash
[root@k8s-master ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
tomcat-deploy-f59fd664-47rbp 0/1 ContainerCreating 0 23s > k8s-node2 > >
tomcat-deploy-f59fd664-8h9s6 0/1 ContainerCreating 0 23s > k8s-node1 > >
tomcat-deploy-f59fd664-fwxwc 0/1 ContainerCreating 0 23s > k8s-node2 > >
将tomcat添加至ingress-nginx中
vi ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: default
annotations:
kubernets.io/ingress.class: "nginx"
spec:
rules:
- host: tomcat.soft.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
通过kubectl将其部署到K8S集群：
kubectl apply -f ingress-tomcat.yaml
添加域名解析及访问服务
C:\Windows\System32\drivers\etc
192.168.200.130 myapp.soft.com
192.168.200.130 tomcat.soft.com
浏览器访问域名
http://tomcat.soft.com:32080/
Apache Tomcat/7.0.94 界面，就不放图了
1.7 对tomcat服务添加https服务
创建私钥和证书
[root@k8s-master ingress-nginx]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
...........+++
..................................+++
e is 65537 (0x10001)
创建自签证书
openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=tomcat.soft.com #注意域名要和服务的域名一致
注意生成的证书是不能直接贴到nginx的pod中去的，我们需要把它先转成特殊格式，这个特殊格式叫secret，它也是标准的k8s集群对象，它可以直接注入到pod中被pod所引用。因此接下来我们需要把它做成secret
kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
查看secret
[root@k8s-master ingress-nginx]# kubectl get secret
NAME TYPE DATA AGE
default-token-pj9g7 kubernetes.io/service-account-token 3 17d
kindly-robin-mychart-token-54rjt kubernetes.io/service-account-token 3 20h
mysecret Opaque 2 3d17h
tomcat-ingress-secret kubernetes.io/tls 2 53s
[root@k8s-master ingress-nginx]# kubectl describe secret tomcat-ingress-secret
Name: tomcat-ingress-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.key: 1679 bytes
tls.crt: 1289 bytes
将证书应用至tomcat服务中
vi ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat-tls
namespace: default
annotations:
kubernets.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- tomcat.soft.com #与secret证书的域名需要保持一致
secretName: tomcat-ingress-secret #secret证书的名称
rules:
- host: tomcat.soft.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
通过kubectl将其部署到K8S集群：
kubectl apply -f ingress-tomcat-tls.yaml
查看ingress
[root@k8s-master ingress-nginx]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-myapp <none> myapp.soft.com 80 46m
ingress-tomcat <none> tomcat.soft.com 80 13m
ingress-tomcat-tls <none> tomcat.soft.com 80, 443 4m4s
浏览器访问域名
https://tomcat.soft.com:32443/
Apache Tomcat/7.0.94 界面，就不放图了