K8S集群通信原理架构
K8S集群通信原理架构
- 访问kubenretes中的Pod和Serivce
- 概念
-
- Pod的四种通信@1
- Pod的四种通信@2
- 通讯模式 @3
- 实验
-
- 本地通信
- 跨节点Pod通信
-
- ping
- telnet
- Nginx-busybox部署
访问kubenretes中的Pod和Serivce
访问kubenretes中的Pod和Serivce的集中方式，包括如下几种：
hostNetwork
hostPort
NodePort
LoadBalancer
Ingress
-
Kubernetes Pod 外部访问
-
在Pod中访问外部网络
-
POD
-
图解Kubernetes网络流量流转路径
概念
Pod的四种通信@1
Pod的四种通信可以分为以下几种情况：
-
本地通信：即同一个Pod内的容器之间进行通信。在同一个Pod内的所有容器共享相同的网络命名空间和IPC（Inter-Process Communication，进程间通信）命名空间，它们可以直接使用localhost或者127.0.0.1来进行通信。这种通信方式不需要经过网络层，因此具有最低的延迟。
-
同节点Pod通信：当多个Pod被调度到同一个节点上时，它们可以通过节点内部的网络进行通信。每个节点都有一个专用的网络接口，用于与其他节点和外部网络进行通信。Pod之间可以通过节点的IP地址相互访问，这种通信方式通常用于在同一节点上的不同Pod之间共享资源或进行协作任务。
-
跨节点Pod通信：当多个Pod被调度到不同的节点上时，它们需要跨越网络进行通信。在Kubernetes中，这种通信通常通过集群级别的网络解决方案（如Flannel、Calico等）来实现。这些解决方案会为每个Pod分配一个独立的IP地址，并通过路由等机制，将不同节点上的Pod连接起来，以实现跨节点的通信。
-
Pod和Pod之外网络实体的通信：Pod可能需要与集群外部的网络实体进行通信，例如其他集群、云服务或外部用户。为了实现这种通信，可以使用Kubernetes Service来暴露Pod内部的服务，并通过负载均衡、端口转发等机制将请求路由到目标Pod。此外，也可以通过Ingress Controller来管理入站流量，将外部请求动态分配给相应的Pod。
需要注意的是，以上四种通信方式都是在Pod级别进行的，而不是容器级别。在同一个Pod内部的容器之间进行通信时，可以直接使用localhost，而对于不同Pod之间的通信，则要依赖于网络层的支持。
Pod的四种通信@2
分别是本地通信、同节点Pod通信、跨节点Pod通信，以及Pod和Pod之外网络实体的通信。
本地通信说的是Pod内部不同容器之间的通信。因为Pod内网容器共享同一个网络协议栈，所以它们之间的通信可以通过Loopback设备完成。
同节点Pod之间的通信，是Cni0虚拟网桥内部的通信，相当于一个二层局域网内部设备通信。
跨节点Pod通信略微复杂一点，但也很直观，发送端数据包通过Cni0网桥的网关流转到节点上，然后经过节点eth0发送给VPC路由。这不会经过任何封包操作。当VPC路由收到数据包时，它通过查询路由表，确认数据包目的地，并把数据包发送给对应的ECS节点。而进入节点之后，因为Flanneld在节点上创建了Cni0的路由，所以数据包会被发送到目的地的Cni0局域网，再到目的地Pod。
最后一种情况，Pod与非Pod网络的实体通信，需要经过节点上的iptables规则做源地址转换，而此规则就是Flanneld依据命令行–ip-masq选项做的配置。
通信原理
通讯模式 @3
K8S！之Pod概念与网络通讯方式详解！
实验
本地通信
tunl0（或称为tunnel0）是一个虚拟网络接口，用于在计算机网络中创建隧道连接。它的作用是允许将数据包从一个网络传输到另一个网络，通过在不同的网络层之间建立逻辑通道。以下是一些 tunl0 网卡的常见用途：
-
隧道连接：tunl0 可以用于创建隧道，将两个远程网络连接起来。这对于连接位于不同地理位置的局域网或虚拟私有网络(VPN)非常有用。
-
加密和安全性：tunl0 可以与其他网络设备结合使用，例如VPN服务器，以提供加密和安全性功能。它可以确保通过隧道传输的数据包在传输过程中得到保护，防止未经授权的访问或窃听。
-
路由器功能：某些路由器可以使用tunl0来实现一些特定的网络功能，如网络隔离、流量分流或负载均衡等。
-
IPv6转换：tunl0 可以用于IPv4和IPv6之间的转换。通过配置tunl0接口，可以实现IPv4到IPv6的转换，使得IPv6数据包能够在IPv4网络中传输。
请注意，tunl0 的具体用途可能因网络配置和需求而有所不同。它的功能取决于如何配置和使用这个虚拟网络接口。
在容器中，tunl0 网卡是一种虚拟网络设备，具有以下作用：
-
隔离网络：tunl0 网卡可以在容器内创建一个虚拟的隔离网络环境，使容器内部的网络流量与宿主机或其他容器之间的网络流量相互隔离。
-
网络通信：通过 tunl0 网卡，容器可以进行网络通信，发送和接收数据包。这使得容器能够与其他容器、宿主机以及外部网络进行通信。
-
VPN 功能：tunl0 网卡可以用于实现虚拟私有网络（VPN）功能，在容器中将网络通信加密并通过安全通道传输，提供了对容器内部数据的保护。
需要注意的是，tunl0 网卡只是其中一种可能的虚拟网络设备，具体使用哪种设备取决于容器运行时或网络管理工具的配置。
跨节点Pod通信
Cilium Vxlan 跨节点通信过程
ping
tcpdump -i any
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
09:58:36.012815 IP 245.0.0.35 > 172-30-30-75: ICMP echo request, id 19, seq 0, length 64
09:58:36.012867 ARP, Request who-has gateway tell 172-30-30-75, length 28
09:58:36.012878 ARP, Reply gateway is-at e6:02:98:0e:de:62 (oui Unknown), length 28
09:58:36.012883 IP 172-30-30-75 > 245.0.0.35: ICMP echo reply, id 19, seq 0, length 64
09:58:36.016353 IP 172-30-30-75.50901 > 172-30-30-75.domain: 29041+ PTR? 59.1.0.245.in-addr.arpa. (41)
09:58:36.185165 IP 172-30-30-75.domain > 172-30-30-75.50901: 29041 NXDomain 0/1/0 (109)
09:58:36.187454 IP 172-30-30-75.53230 > 172-30-30-75.domain: 40801+ PTR? 35.0.0.245.in-addr.arpa. (41)
09:58:36.343923 IP 172-30-30-75.domain > 172-30-30-75.53230: 40801 NXDomain 0/1/0 (109)
09:58:36.344870 IP 172-30-30-75.41486 > 172-30-30-75.domain: 28711+ PTR? 221.1.0.245.in-addr.arpa. (42)
09:58:36.503676 IP 172-30-30-75.domain > 172-30-30-75.41486: 28711 NXDomain 0/1/0 (110)
09:58:37.013066 IP 245.0.0.35 > 172-30-30-75: ICMP echo request, id 19, seq 1, length 64
09:58:37.013130 IP 172-30-30-75 > 245.0.0.35: ICMP echo reply, id 19, seq 1, length 64
09:58:38.013194 IP 245.0.0.35 > 172-30-30-75: ICMP echo request, id 19, seq 2, length 64
09:58:38.013280 IP 172-30-30-75 > 245.0.0.35: ICMP echo reply, id 19, seq 2, length 64
09:58:39.013647 IP 245.0.0.35 > 172-30-30-75: ICMP echo request, id 19, seq 3, length 64
09:58:39.013696 IP 172-30-30-75 > 245.0.0.35: ICMP echo reply, id 19, seq 3, length 64
09:58:40.015328 IP 245.0.0.35 > 172-30-30-75: ICMP echo request, id 19, seq 4, length 64
09:58:40.015390 IP 172-30-30-75 > 245.0.0.35: ICMP echo reply, id 19, seq 4, length 64
telnet
容器抓包
tcpdump -i any
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
10:05:37.352558 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 1234779716:1234779718, ack 1392392637, win 507, options [nop,nop,TS val 4039653910 ecr 1521909118], length 2: HTTP
10:05:37.352664 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 2, win 85, options [nop,nop,TS val 1521942307 ecr 4039653910], length 0
10:05:37.353485 IP 172-30-30-75.50607 > 172-30-30-75.domain: 52741+ PTR? 59.1.0.245.in-addr.arpa. (41)
10:05:37.353925 IP 172-30-30-75.domain > 172-30-30-75.50607: 52741 NXDomain 0/0/0 (41)
10:05:37.355976 IP 172-30-30-75.33514 > 172-30-30-75.domain: 49331+ PTR? 35.0.0.245.in-addr.arpa. (41)
10:05:37.356127 IP 172-30-30-75.domain > 172-30-30-75.33514: 49331 NXDomain 0/0/0 (41)
10:05:38.713209 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 2:4, ack 1, win 507, options [nop,nop,TS val 4039655271 ecr 1521942307], length 2: HTTP
10:05:38.713236 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 4, win 85, options [nop,nop,TS val 1521943668 ecr 4039655271], length 0
10:05:38.904040 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 4:6, ack 1, win 507, options [nop,nop,TS val 4039655462 ecr 1521943668], length 2: HTTP
10:05:38.904076 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 6, win 85, options [nop,nop,TS val 1521943858 ecr 4039655462], length 0
10:05:39.074127 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 6:8, ack 1, win 507, options [nop,nop,TS val 4039655632 ecr 1521943858], length 2: HTTP
10:05:39.074170 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 8, win 85, options [nop,nop,TS val 1521944029 ecr 4039655632], length 0
10:05:41.029883 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 8:10, ack 1, win 507, options [nop,nop,TS val 4039657587 ecr 1521944029], length 2: HTTP
10:05:41.029914 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 10, win 85, options [nop,nop,TS val 1521945984 ecr 4039657587], length 0
10:05:42.505790 ARP, Request who-has gateway tell 172-30-30-75, length 28
10:05:42.505955 ARP, Reply gateway is-at e6:02:98:0e:de:62 (oui Unknown), length 28
10:05:42.506394 IP 172-30-30-75.34241 > 172-30-30-75.domain: 52238+ PTR? 221.1.0.245.in-addr.arpa. (42)
10:05:42.506747 IP 172-30-30-75.domain > 172-30-30-75.34241: 52238 NXDomain 0/0/0 (42)
10:05:42.814118 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [P.], seq 10:13, ack 1, win 507, options [nop,nop,TS val 4039659368 ecr 1521945984], length 3: HTTP
10:05:42.814148 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.814338 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [P.], seq 1:310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 309: HTTP: HTTP/1.1 400 Bad Request
10:05:42.814517 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [F.], seq 310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.816323 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [.], ack 310, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816330 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [.], ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816357 IP 245.0.0.35.37762 > 172-30-30-75.http: Flags [F.], seq 13, ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816415 IP 172-30-30-75.http > 245.0.0.35.37762: Flags [.], ack 14, win 85, options [nop,nop,TS val 1521947771 ecr 4039659373], length 0
10:05:50.607206 IP 245.0.0.35.39334 > 172-30-30-75.http: Flags [S], seq 2866573496, win 64860, options [mss 1410,sackOK,TS val 4039667165 ecr 0,nop,wscale 7], length 0
10:05:50.607278 IP 172-30-30-75.http > 245.0.0.35.39334: Flags [S.], seq 2046962811, ack 2866573497, win 43338, options [mss 1410,sackOK,TS val 1521955562 ecr 4039667165,nop,wscale 9], length 0
10:05:50.607846 IP 245.0.0.35.39334 > 172-30-30-75.http: Flags [.], ack 1, win 507, options [nop,nop,TS val 4039667165 ecr 1521955562], length 0
10:06:50.664902 IP 172-30-30-75.http > 245.0.0.35.39334: Flags [F.], seq 1, ack 1, win 85, options [nop,nop,TS val 1522015619 ecr 4039667165], length 0
10:06:50.675057 IP 245.0.0.35.39334 > 172-30-30-75.http: Flags [F.], seq 1, ack 2, win 507, options [nop,nop,TS val 4039727231 ecr 1522015619], length 0
10:06:50.675094 IP 172-30-30-75.http > 245.0.0.35.39334: Flags [.], ack 2, win 85, options [nop,nop,TS val 1522015629 ecr 4039727231], length 0
10:06:55.721822 ARP, Request who-has gateway tell 172-30-30-75, length 28
10:06:55.722010 ARP, Reply gateway is-at e6:02:98:0e:de:62 (oui Unknown), length 28
宿主机抓包
tcpdump -i any host 245.0.0.35
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
10:05:37.352355 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 1234779716:1234779718, ack 1392392637, win 507, options [nop,nop,TS val 4039653910 ecr 1521909118], length 2: HTTP
10:05:37.352552 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 0:2, ack 1, win 507, options [nop,nop,TS val 4039653910 ecr 1521909118], length 2: HTTP
10:05:37.352667 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 2, win 85, options [nop,nop,TS val 1521942307 ecr 4039653910], length 0
10:05:37.352713 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 2, win 85, options [nop,nop,TS val 1521942307 ecr 4039653910], length 0
10:05:38.713122 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 2:4, ack 1, win 507, options [nop,nop,TS val 4039655271 ecr 1521942307], length 2: HTTP
10:05:38.713206 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 2:4, ack 1, win 507, options [nop,nop,TS val 4039655271 ecr 1521942307], length 2: HTTP
10:05:38.713238 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 4, win 85, options [nop,nop,TS val 1521943668 ecr 4039655271], length 0
10:05:38.713272 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 4, win 85, options [nop,nop,TS val 1521943668 ecr 4039655271], length 0
10:05:38.903911 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 4:6, ack 1, win 507, options [nop,nop,TS val 4039655462 ecr 1521943668], length 2: HTTP
10:05:38.904036 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 4:6, ack 1, win 507, options [nop,nop,TS val 4039655462 ecr 1521943668], length 2: HTTP
10:05:38.904078 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 6, win 85, options [nop,nop,TS val 1521943858 ecr 4039655462], length 0
10:05:38.904115 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 6, win 85, options [nop,nop,TS val 1521943858 ecr 4039655462], length 0
10:05:39.073997 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 6:8, ack 1, win 507, options [nop,nop,TS val 4039655632 ecr 1521943858], length 2: HTTP
10:05:39.074122 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 6:8, ack 1, win 507, options [nop,nop,TS val 4039655632 ecr 1521943858], length 2: HTTP
10:05:39.074173 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 8, win 85, options [nop,nop,TS val 1521944029 ecr 4039655632], length 0
10:05:39.074216 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 8, win 85, options [nop,nop,TS val 1521944029 ecr 4039655632], length 0
10:05:41.029800 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 8:10, ack 1, win 507, options [nop,nop,TS val 4039657587 ecr 1521944029], length 2: HTTP
10:05:41.029880 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 8:10, ack 1, win 507, options [nop,nop,TS val 4039657587 ecr 1521944029], length 2: HTTP
10:05:41.029917 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 10, win 85, options [nop,nop,TS val 1521945984 ecr 4039657587], length 0
10:05:41.029959 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 10, win 85, options [nop,nop,TS val 1521945984 ecr 4039657587], length 0
10:05:42.814004 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 10:13, ack 1, win 507, options [nop,nop,TS val 4039659368 ecr 1521945984], length 3: HTTP
10:05:42.814114 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [P.], seq 10:13, ack 1, win 507, options [nop,nop,TS val 4039659368 ecr 1521945984], length 3: HTTP
10:05:42.814151 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.814193 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.814347 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [P.], seq 1:310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 309: HTTP: HTTP/1.1 400 Bad Request
10:05:42.814419 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [P.], seq 1:310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 309: HTTP: HTTP/1.1 400 Bad Request
10:05:42.814521 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [F.], seq 310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.814556 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [F.], seq 310, ack 13, win 85, options [nop,nop,TS val 1521947769 ecr 4039659368], length 0
10:05:42.816214 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [.], ack 310, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816320 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [.], ack 310, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816252 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [.], ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816328 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [.], ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816260 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [F.], seq 13, ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816355 IP 245.0.0.35.37762 > 245.0.1.59.http: Flags [F.], seq 13, ack 311, win 505, options [nop,nop,TS val 4039659373 ecr 1521947769], length 0
10:05:42.816417 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 14, win 85, options [nop,nop,TS val 1521947771 ecr 4039659373], length 0
10:05:42.816445 IP 245.0.1.59.http > 245.0.0.35.37762: Flags [.], ack 14, win 85, options [nop,nop,TS val 1521947771 ecr 4039659373], length 0
10:05:50.607034 IP 245.0.0.35.39334 > 245.0.1.59.http: Flags [S], seq 2866573496, win 64860, options [mss 1410,sackOK,TS val 4039667165 ecr 0,nop,wscale 7], length 0
10:05:50.607202 IP 245.0.0.35.39334 > 245.0.1.59.http: Flags [S], seq 2866573496, win 64860, options [mss 1410,sackOK,TS val 4039667165 ecr 0,nop,wscale 7], length 0
10:05:50.607282 IP 245.0.1.59.http > 245.0.0.35.39334: Flags [S.], seq 2046962811, ack 2866573497, win 43338, options [mss 1410,sackOK,TS val 1521955562 ecr 4039667165,nop,wscale 9], length 0
10:05:50.607311 IP 245.0.1.59.http > 245.0.0.35.39334: Flags [S.], seq 2046962811, ack 2866573497, win 43338, options [mss 1410,sackOK,TS val 1521955562 ecr 4039667165,nop,wscale 9], length 0
10:05:50.607800 IP 245.0.0.35.39334 > 245.0.1.59.http: Flags [.], ack 1, win 507, options [nop,nop,TS val 4039667165 ecr 1521955562], length 0
10:05:50.607844 IP 245.0.0.35.39334 > 245.0.1.59.http: Flags [.], ack 1, win 507, options [nop,nop,TS val 4039667165 ecr 1521955562], length 0
Nginx-busybox部署
cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ssh-deployment
spec:
replicas: 2
selector:
matchLabels:
app: nginx-ssh
template:
metadata:
labels:
app: nginx-ssh
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-ssh
topologyKey: "kubernetes.io/hostname"
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
- name: ssh
image: busybox
imagePullPolicy: IfNotPresent
ports:
- containerPort: 81
args:
- /bin/sh
- -c
- sleep 36000