K8S-集群添加worker节点脚本

挂载磁盘

可能已经有/export了

那就先停止docker:

1、systemctl restart docker

2、systemctl stop docker  systemctl status docker

4、执行下面的

查看磁盘使用情况
#df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb        2.0T   81M  1.9T   1% /export

如果磁盘没挂载，需要挂载/export
DISK_VOLUMN=/dev/vdb
mkfs.xfs -n ftype=1 -f $DISK_VOLUMN
DISK_UUID=`blkid -s UUID $DISK_VOLUMN | awk '{print $2}' | sed  "s/\"//g"`
echo "$DISK_UUID /export xfs defaults,pquota 0 0" >> /etc/fstab
mkdir /export
mount -a

下载k8s依赖的相关docker镜像

略

修改docker配置

# 改配置
vim /etc/docker/daemon.json
{
  "insecure-registries": ["xx.xx.xx.xx:80","dockerhub.xx.xx.local"],
  "data-root": "/export/Data/docker",
  "exec-opts": ["native.cgroupdriver=systemd"],
  "default-runtime": "nvidia",
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

#重启docker
systemctl daemon-reload
systemctl restart docker

设置时间同步

crontab -e
0 */1 * * * /usr/sbin/ntpdate time1.aliyun.com

 

GPU显卡安装

# 查看是否已关闭
lsmod | grep nouveau

创建脚本：
nouveau.sh

#!/bin/bash  

# 创建文件  
vim /usr/lib/modprobe.d/dist-blacklist.conf 
# 写入内容
blacklist nouveaua
options nouveau modeset=0
# 备份
mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r)-nouveau.img.old
# 删除
dracut /boot/initramfs-$(uname -r).img $(uname -r)



# 驱动加入黑名单中:
# 在/etc/modprobe.d/blacklist.conf 在后面加入：blacklist nouveau (若文件不存在则新建文件，将内容写入文件内)

# 备份 the initramfs file ：
mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak

# 重新建立 the initramfs file：
dracut -v /boot/initramfs-$(uname -r).img $(uname -r)



# 重启
reboot

# 查看是否已关闭
lsmod | grep nouveau

# 安装开发工具集 # 安装内核头文件 # 安装gcc
yum groupinstall "Development Tools" -y
yum install "kernel-devel-uname-r == $(uname -r)" -y
yum install gcc-c++ -y

cd /export/v1.21.0/gpu
rpm -ivh --force kernel-3.10.0-1160.88.1.el7.x86_64.rpm
rpm -ivh --force kernel-headers-3.10.0-1160.88.1.el7.x86_64.rpm
rpm -ivh --force kernel-devel-3.10.0-1160.88.1.el7.x86_64.rpm
rpm -ivh --force kernel-tools-3.10.0-1160.88.1.el7.x86_64.rpm
yum install libglvnd-devel -y
./NVIDIA-Linux-x86_64-550.54.15.run --kernel-source-path=/usr/src/kernels/3.10.0-1160.88.1.el7.x86_64



# 写脚本 docker nvidia 容器
cd /export/v1.21.0/gpu
unzip nvidia-container-runtime.zip
cd nvidia-container-runtime
rpm -Uvh *.rpm --nodeps --force
systemctl daemon-reload
systemctl restart docker

work节点K8S安装

cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum -y install --setopt=obsoletes=0 kubeadm-1.21.0-0  kubelet-1.21.0-0 kubectl-1.21.0-0

# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"


设置kubelet为开机自启动即可，由于没有生成配置文件，集群初始化后自动启动
# systemctl enable kubelet

添加节点

1、子节点执行：
kubeadm reset 

2、master生成新节点加入的命令：kubeadm token create --print-join-command
kubeadm join xx.xx.xx.xx:6443 --token xxx --discovery-token-ca-cert-hash sha256:xxx

3、resboot一下子节点:

4、添加标签：master执行
kubectl label nodes machine-001  node-role.kubernetes.io/worker=
kubectl label nodes machine-001  node-app-type=xxx
kubectl label nodes machine-001  node-machine-type=gpu
kubectl label nodes machine-001  node-module-type=xxx_xxx

如需对节点重新达标，需要 --overwrite
kubectl label nodes machine-001  node-machine-type=cpu --overwrite

检查：

master执行：

kubectl describe node 节点名称；

查看集群现状[包括标签]

 kubectl get node --show-labels

给要添加的机器编号

ip1
hostnamectl set-hostname k8s-worker-01
ip2
hostnamectl set-hostname k8s-worker-02
ip3
hostnamectl set-hostname k8s-worker-03

修改 /etc/hosts

#k8s
ip1  master-01
ip2  k8s-worker-01
ip3  k8s-worker-02
ip4  k8s-worker-03

升级最新docker版本[需要有外网]

yum install -y yum-utils  device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce -y


systemctl start docker
systemctl enable docker
docker version

驱逐节点

卸载 k8s
登录Master机器执行：
# 节点驱逐
kubectl drain k8s-worker-06 --delete-local-data --force --ignore-daemonsets
# 删除节点
kubectl delete node k8s-worker-06

登录需要卸载的机器IP 然后执行
kubeadm reset

iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
ipvsadm --clear
systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/*
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/*
rm -rf $HOME/.kube/config
systemctl start docker

去集群各机器，编辑/etc/hosts，删除被驱逐的节点hosts配置。