部署work节点基础环境

注1

本套环境三个master节点同时也做work节点,因此基础环境不需要再做部署.

如果添加新的work节点,参考基础环境准备即可

注2

master节点运行的组件有:kubectl,etcd,flannel,kube-apiserver,kube-controller-manager,kube-scheduler,docker.

work节点运行的组件有:flannel,docker,kubelet,kube-proxy.
flannel和docker已经部署好,只需要部署kubelet和kube-proxy即可

注3

flannel的网络需要和docker网络在同一网段

所有节点都需要修改docker配置文件

[root@k8s-node1 kube-scheduler]#  systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2019-11-04 20:43:37 EST; 2h 53min ago
     Docs: https://docs.docker.com
 Main PID: 1641 (dockerd)
    Tasks: 11
   Memory: 146.9M
   CGroup: /system.slice/docker.service
           └─1641 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Nov 04 20:43:31 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:31.650071753-05:00" level=info msg="ccResolverWrapper: sending updat...ule=grpc
Nov 04 20:43:31 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:31.650086343-05:00" level=info msg="ClientConn switching balancer to...ule=grpc
Nov 04 20:43:31 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:31.831923167-05:00" level=info msg="[graphdriver] using prior storag...verlay2"
Nov 04 20:43:32 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:32.278737358-05:00" level=info msg="Loading containers: start."
Nov 04 20:43:34 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:34.889127744-05:00" level=info msg="Default bridge (docker0) is assi...address"
Nov 04 20:43:36 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:36.265732048-05:00" level=info msg="Loading containers: done."
Nov 04 20:43:37 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:37.279426242-05:00" level=info msg="Docker daemon" commit=9013bf583a...=19.03.4
Nov 04 20:43:37 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:37.280331422-05:00" level=info msg="Daemon has completed initialization"
Nov 04 20:43:37 k8s-node1 systemd[1]: Started Docker Application Container Engine.
Nov 04 20:43:37 k8s-node1 dockerd[1641]: time="2019-11-04T20:43:37.548397588-05:00" level=info msg="API listen on /var/run/docker.sock"
Hint: Some lines were ellipsized, use -l to show in full.
[root@k8s-node1 kube-scheduler]#

修改/usr/lib/systemd/system/docker.service这个文件

默认文件见下

[root@k8s-node1 kube-scheduler]#  cat /usr/lib/systemd/system/docker.service |grep -v ^# |grep -v ^$
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
[root@k8s-node1 kube-scheduler]#

修改后见下

[root@k8s-node1 kube-scheduler]#  cat /usr/lib/systemd/system/docker.service |grep -v ^# |grep -v ^$
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
EnvironmentFile=/run/flannel/docker
ExecStart=/usr/bin/dockerd --log-level=error $DOCKER_NETWORK_OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
[root@k8s-node1 kube-scheduler]#

$DOCKER_NETWORK_OPTIONS.dockerd 运行时会调用其它 docker 命令,如 docker-proxy,所以需要将 docker 命令所在的目录加到 PATH 环境变量中.

flanneld 启动时将网络配置写入 /run/flannel/docker 文件中,dockerd 启动前读取该文件中的环境变量 DOCKER_NETWORK_OPTIONS,然后设置 docker0 网桥网段.

如果指定了多个 EnvironmentFile 选项,则必须将 /run/flannel/docker 放在最后(确保 docker0 使用 flanneld 生成的 bip 参数).

改好后,重启docker服务

systemctl daemon-reload && systemctl restart docker

flannel1.1和docker0在同一个网段内

3: flannel.1:  mtu 1450 qdisc noqueue state UNKNOWN group default 
    link/ether da:52:0c:1f:81:d3 brd ff:ff:ff:ff:ff:ff
    inet 172.30.86.0/32 scope global flannel.1
       valid_lft forever preferred_lft forever
    inet6 fe80::d852:cff:fe1f:81d3/64 scope link 
       valid_lft forever preferred_lft forever
4: docker0:  mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:b8:ec:6d:b4 brd ff:ff:ff:ff:ff:ff
    inet 172.30.86.1/24 brd 172.30.86.255 scope global docker0
       valid_lft forever preferred_lft forever