k8s——kubernetes二进制多节点部署（单节点部署为基础）
这里写目录标题
- 1. kubernetes集群架构与组件
-
- 1.1 Master组件
-
- 1.1.1 kube-apiserver
- 1.1.2 kube-controller-manager
- 1.1.3 kube-scheduler
- 1.1.4 etcd
- 1.2 Node组件
-
- 1.2.1 kubelet
- 1.2.2 kube-proxy
- 1.2.3 docker或rocket
- 2.kubernetes集群部署
-
- 2.1 部署过程
- 2.2 实验拓扑图
- 3.二进制多节点部署
-
- 3.1 实验平台环境规划
- 3.2 实验步骤
-
- 1.K8S单节点部署
- 2.多节点部署
1. kubernetes集群架构与组件
1.1 Master组件
1.1.1 kube-apiserver
Kubernetes API,集群的统- . 入口，各组件协调者，以RESTful API提供接口服务，所有对象资源的增删改查和监听操作都交给APIServer处理后再提交给Etcd存储。
1.1.2 kube-controller-manager
处理集群中常规后台任务，- - 个资源对应一个控制器， 而ControllerManager就是负贵管理这些控制器的。
1.1.3 kube-scheduler
根据调度算法为新创建的Pod选择- - 个Node节点，可以任意部署可以部署在同一个节点上,也可以部署在不同的节点上。
1.1.4 etcd
分布式键值存储系统。用于保存集群状态数据，比如Pod. Service等对象信息。
1.2 Node组件
1.2.1 kubelet
kubelet是Master在Node节点上的Agent,管理本机运行容器的生命周期，比如创建容器、Pod挂载数据卷、下 载secret、获取容器和节点状态等工作。kubelet将每个Pod转换成一组容器。
1.2.2 kube-proxy
在Node节点上实现Pod网络代理，维护网络规则和四层负载均衡工作。
1.2.3 docker或rocket
容器引擎，运行容器。
2.kubernetes集群部署
2.1 部署过程
1.官方提供的三种部署方式
2. Kubernetes 平台环境规划
3.自签SSL证书
4.Etcd数据库集群部署
5. Node安装Docker
6. Flannel容 器集群网络部署
7.部署Master组件
8.部署Node组件
9.部署-一个测试示例
10.部署Web UI ( Dashboard )
11.部署集群内部DNS解析服务(CoreDNS)
2.2 实验拓扑图
自签SSL证书
3.二进制多节点部署
3.1 实验平台环境规划
master1：192.168.200.100
master2：192.168.200.90
node1:192.168.200.110
node2:192.168.200.120
load balance(master)192.168.200.80
load balance(back up)192.169.200.70
vip : 192.168.200.200
3.2 实验步骤
1.K8S单节点部署
master01 192.168.200.100
[root@localhost ~]# iptables -F
[root@localhost ~]# setenforce 0
[root@localhost ~]# hostnamectl set-hostname master
[root@localhost ~]# su
[root@master01 ~]# mkdir k8s
[root@master01 ~]# cd k8s/
[root@master01 k8s]# rz -E
rz waiting to receive.
[root@master01 k8s]# ls
etcd-cert.sh etcd.sh
[root@master01 k8s]# mkdir etcd-cert
[root@master01 k8s]# ls
etcd-cert etcd-cert.sh etcd.sh
[root@master01 k8s]# mv etcd-cert.sh etcd-cert
[root@master01 k8s]# ls
etcd-cert etcd.sh
[root@master01 k8s]# cd /usr/local/bin/
[root@master01 bin]# ls
[root@master01 bin]# rz -E
rz waiting to receive.
[root@master01 bin]# ls
cfssl cfssl-certinfo cfssljson
[root@master01 bin]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
[root@master01 bin]# ls
cfssl cfssl-certinfo cfssljson #cfssl 生成证书工具 cfssljson通过传入json文件生成证书 cfssl-certinfo查看证书信息
#开始制作证书
[root@master01 bin]# cd ~/k8s/etcd-cert/
#定义ca证书
cat > ca-config.json <{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
[root@master01 etcd-cert]# ls
ca-config.json etcd-cert.sh
#实现证书签名
cat > ca-csr.json <{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
[root@master01 etcd-cert]# ls
ca-config.json ca-csr.json etcd-cert.sh
#生产证书，生成ca-key.pem ca.pem
[root@master01 etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2020/10/08 00:46:06 [INFO] generating a new CA key and certificate from CSR
2020/10/08 00:46:06 [INFO] generate received request
2020/10/08 00:46:06 [INFO] received CSR
2020/10/08 00:46:06 [INFO] generating key: rsa-2048
2020/10/08 00:46:07 [INFO] encoded CSR
2020/10/08 00:46:07 [INFO] signed certificate with serial number 682011236265898836699745690623627317797100291414
[root@master01 etcd-cert]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem etcd-cert.sh
#指定etcd三个节点之间的通信验证
cat > server-csr.json <{
"CN": "etcd",
"hosts": [
"192.168.200.100",
"192.168.200.110",
"192.168.200.120"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
EOF
[root@master01 etcd-cert]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem etcd-cert.sh server-csr.json
#生成ETCD证书 server-key.pem server.pem
[root@master01 etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
2020/10/08 00:47:56 [INFO] generate received request
2020/10/08 00:47:56 [INFO] received CSR
2020/10/08 00:47:56 [INFO] generating key: rsa-2048
2020/10/08 00:47:57 [INFO] encoded CSR
2020/10/08 00:47:57 [INFO] signed certificate with serial number 622675374353910529509628185945252518303577262750
2020/10/08 00:47:57 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master01 etcd-cert]# ls
ca-config.json ca-csr.json ca.pem server.csr server-key.pem
ca.csr ca-key.pem etcd-cert.sh server-csr.json server.pem
#下载ETCD 二进制包地址
[root@master01 etcd-cert]# cd ~/k8s/
[root@master01 k8s]# ls
etcd-cert etcd-v3.3.10-linux-amd64.tar.gz kubernetes-server-linux-amd64.tar.gz
etcd.sh flannel-v0.10.0-linux-amd64.tar.gz
[root@master01 k8s]# tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
[root@master01 k8s]# ls etcd-v3.3.10-linux-amd64
Documentation etcd etcdctl README-etcdctl.md README.md READMEv2-etcdctl.md
[root@master01 k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p #创建etcd配置文件，命令文件，证书
[root@master01 k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/
#证书拷贝
[root@master01 k8s]# cp etcd-cert/*.pem /opt/etcd/ssl/
[root@master01 k8s]# ls /opt/etcd/ssl/
ca-key.pem ca.pem server-key.pem server.pem
#进入卡住状态等待其他节点加入
[root@master01 k8s]# vim etcd.sh
#!/bin/bash
# example: ./etcd.sh etcd01 192.168.1.10 etcd02=https://192.168.1.11:2380,etcd03=https://192.168.1.12:2380
ETCD_NAME=$1
ETCD_IP=$2
ETCD_CLUSTER=$3
WORK_DIR=/opt/etcd
cat <$WORK_DIR/cfg/etcd
#[Member]
ETCD_NAME="${ETCD_NAME}"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
cat </usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=${WORK_DIR}/cfg/etcd
ExecStart=${WORK_DIR}/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=\${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=${WORK_DIR}/ssl/server.pem \
--key-file=${WORK_DIR}/ssl/server-key.pem \
--peer-cert-file=${WORK_DIR}/ssl/server.pem \
--peer-key-file=${WORK_DIR}/ssl/server-key.pem \
--trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
--peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd
[root@master01 k8s]# bash etcd.sh etcd01 192.168.200.100 etcd02=https://192.168.200.110:2380,etcd03=https://192.168.200.120:2380
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
#打开另外一个会话，会发现etcd进程已经开启
[root@master01 ~]# ps -ef | grep etcd
root 13066 12731 0 00:59 pts/1 00:00:00 bash etcd.sh etcd01 192.168.200.100 etcd02=https://192.168.200.110:2380,etcd03=https://192.168.200.120:2380
root 13113 13066 0 00:59 pts/1 00:00:00 systemctl restart etcd
root 13119 1 4 00:59 ? 00:00:02 /opt/etcd/bin/etcd --name=etcd01 --data-dir=/var/lib/etcd/default.etcd --listen-peer-urls=https://192.168.200.100:2380 --listen-client-urls=https://192.168.200.100:2379,http://127.0.0.1:2379 --advertise-client-urls=https://192.168.200.100:2379 --initial-advertise-peer-urls=https://192.168.200.100:2380 --initial-cluster=etcd01=https://192.168.200.100:2380,etcd02=https://192.168.200.110:2380,etcd03=https://192.168.200.120:2380 --initial-cluster-token=etcd-cluster --initial-cluster-state=new --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
root 13192 13149 0 01:00 pts/2 00:00:00 grep --color=auto etcd
#拷贝证书去其他节点
[root@master01 ~]# scp -r /opt/etcd/ [email protected]:/opt/
The authenticity of host '192.168.200.110 (192.168.200.110)' can't be established.
ECDSA key fingerprint is SHA256:eLdHi9BvCNVro0zGiYPq1F+Psfoo9V+9EDIvdZDR8vM.
ECDSA key fingerprint is MD5:2d:34:ae:97:fd:bc:af:4f:e1:6b:92:22:48:4d:69:b4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.200.110' (ECDSA) to the list of known hosts.
[email protected]'s password:
etcd 100% 523 508.0KB/s 00:00
etcd 100% 18MB 85.5MB/s 00:00
etcdctl 100% 15MB 95.2MB/s 00:00
ca-key.pem 100% 1679 2.4MB/s 00:00
ca.pem 100% 1265 1.4MB/s 00:00
server-key.pem 100% 1679 651.0KB/s 00:00
server.pem 100% 1338 737.7KB/s 00:00
[root@master01 ~]# scp -r /opt/etcd/ [email protected]:/opt/
The authenticity of host '192.168.200.120 (192.168.200.120)' can't be established.
ECDSA key fingerprint is SHA256:dMvbIJyuN9aFqJR+OwwLY436gqKEgtipcBLofzOilgU.
ECDSA key fingerprint is MD5:7a:d8:f0:f5:c5:ff:95:36:11:fe:e8:b3:c0:dc:d7:2e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.200.120' (ECDSA) to the list of known hosts.
[email protected]'s password:
etcd 100% 523 362.4KB/s 00:00
etcd 100% 18MB 95.3MB/s 00:00
etcdctl 100% 15MB 104.1MB/s 00:00
ca-key.pem 100% 1679 2.2MB/s 00:00
ca.pem 100% 1265 2.1MB/s 00:00
server-key.pem 100% 1679 715.4KB/s 00:00
server.pem 100% 1338 759.8KB/s 00:00
#启动脚本拷贝其他节点
[root@master01 ~]# scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
root@192.168.200.110's password:
etcd.service 100% 923 412.9KB/s 00:00
[root@master01 ~]# scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
[email protected]'s password:
etcd.service 100% 923 805.5KB/s 00:00
启动
[root@master01 k8s]# systemctl start etcd
[root@master01 k8s]# systemctl enable etcd
[root@master01 k8s]# systemctl status etcd
● etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2020-10-08 01:07:12 CST; 1min 44s ago
#检查群集状态
[root@master01 k8s]# cd etcd-cert/ #需要在有证书的文件夹里面验证
[root@master01 etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379" cluster-health
member 6a670a4e5fc9896c is healthy: got healthy result from https://192.168.200.100:2379
member 8f01b24208072c50 is healthy: got healthy result from https://192.168.200.120:2379
member 9f5aa0e1c7d6b024 is healthy: got healthy result from https://192.168.200.110:2379
cluster is healthy
2.配置node01 192.168.200.110
[root@promote ~]# iptables -F
[root@promote ~]# setenforce 0
[root@promote ~]# hostnamectl set-hostname node01
[root@promote ~]# su
#在node01节点修改
[root@node01 ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd02" ##02
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.200.110:2380" ## 110
ETCD_LISTEN_CLIENT_URLS="https://192.168.200.110:2379" ## 110
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.200.110:2380" ## 110
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.200.110:2379" ## 110
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.200.100:2380,etcd02=https://192.168.200.110:2380,etcd03=https://192.168.200.120:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
启动
[root@node01 ~]# systemctl start etcd
[root@node01 ~]# systemctl enable etcd
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
[root@node01 ~]# systemctl status etcd
● etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2020-10-08 01:07:21 CST; 25s ago
3.配置node02 192.168.200.120
[root@localhost ~]# iptables -F
[root@localhost ~]# setenforce 0
[root@localhost ~]# hostnamectl set-hostname node02
[root@localhost ~]# su
#在node02节点修改
[root@node02 ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd03" ## 03
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.200.120:2380" ## 120
ETCD_LISTEN_CLIENT_URLS="https://192.168.200.120:2379" ## 120
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.200.120:2380" ## 120
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.200.120:2379" ## 120
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.200.100:2380,etcd02=https://192.168.200.110:2380,etcd03=https://192.168.200.120:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
~
启动
[root@node02 ~]# systemctl start etcd
[root@node02 ~]# systemctl enable etcd
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
[root@node02 ~]# systemctl status etcd
● etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2020-10-08 01:07:19 CST; 38s ago
4.所有node节点部署docker引擎
1.先卸载原来版本的docker
yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
2.需要的安装包
yum install -y yum-utils device-mapper-persistent-data lvm2
3.设置阿里云镜像
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
4.更新yum软件包索引
yum makecache fast
5.安装docker-CE，安装docker相关的 docker-ce社区 ee企业版
yum -y install docker-ce docker-ce-cli containerd.io
6.启动docker
systemctl start docker
systemctl enable docker
7.设置镜像加速
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://sno1b9w3.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
8.使用docker version 查看是否安装成功
9.网络优化 开启路由功能
vim /etc/sysctl.conf
net.ipv4.ip_forward=1
sysctl -p
重启网络
service network restart
systemctl restart docker
5.配置flannel网络配置
#写入分配的子网段到ETCD中，供flannel使用
[root@master01 etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}
#查看写入的信息
[root@master01 etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379" get /coreos.com/network/config
{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}
#拷贝到所有node节点（只需要部署在node节点即可）
[root@master01 etcd-cert]# cd ~/k8s/
[root@master01 k8s]# ls
etcd-cert etcd-v3.3.10-linux-amd64 flannel-v0.10.0-linux-amd64.tar.gz
etcd.sh etcd-v3.3.10-linux-amd64.tar.gz kubernetes-server-linux-amd64.tar.gz
[root@master01 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz [email protected]:/root
root@192.168.200.110's password:
flannel-v0.10.0-linux-amd64.tar.gz 100% 9479KB 73.2MB/s 00:00
[root@master01 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz [email protected]:/root
[email protected]'s password:
flannel-v0.10.0-linux-amd64.tar.gz 100% 9479KB 58.7MB/s 00:00
#所有node节点操作解压
[root@node01 ~]# tar zxvf flannel-v0.10.0-linux-amd64.tar.gz
flanneld
mk-docker-opts.sh
README.md
[root@node01 ~]# ls
anaconda-ks.cfg initial-setup-ks.cfg 公共 图片 音乐
flanneld mk-docker-opts.sh 模板 文档 桌面
flannel-v0.10.0-linux-amd64.tar.gz README.md 视频 下载
[root@node01 ~]# mkdir /opt/kubernetes/{cfg,bin,ssl} -p
[root@node01 ~]# mv mk-docker-opts.sh flanneld /opt/kubernetes/bin/
[root@node01 ~]# vim flannel.sh
#!/bin/bash
ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"} ##生产环境中指向其中一个master，一般master做etcd服务器
cat </opt/kubernetes/cfg/flanneld
FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
-etcd-cafile=/opt/etcd/ssl/ca.pem \
-etcd-certfile=/opt/etcd/ssl/server.pem \
-etcd-keyfile=/opt/etcd/ssl/server-key.pem"
EOF
cat </usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable flanneld
systemctl restart flanneld
#开启flannel网络功能
[root@node01 ~]# bash flannel.sh https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379
Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.
#配置docker连接flannel
[root@node01 ~]# vim /usr/lib/systemd/system/docker.service
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=/run/flannel/subnet.env ##加这句话
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock ##注释上面一句，增加这一句
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
[root@node01 ~]# cat /run/flannel/subnet.env
DOCKER_OPT_BIP="--bip=172.17.25.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=false"
DOCKER_OPT_MTU="--mtu=1450"
DOCKER_NETWORK_OPTIONS=" --bip=172.17.25.1/24 --ip-masq=false --mtu=1450" #说明：bip指定启动时的子网
#重启docker服务
[root@node01 ~]# systemctl daemon-reload
[root@node01 ~]# systemctl restart docker
[root@node01 ~]# ifconfig
docker0: flags=4099,BROADCAST,MULTICAST> mtu 1500
inet 172.17.25.1 netmask 255.255.255.0 broadcast 172.17.25.255
ether 02:42:cd:34:d8:9f txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.110 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::7fc5:140b:bf58:bfb8 prefixlen 64 scopeid 0x20
ether 00:0c:29:60:ee:f7 txqueuelen 1000 (Ethernet)
RX packets 215434 bytes 175791247 (167.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 119849 bytes 13699087 (13.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.17.25.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::d8fd:c4ff:fe97:35c0 prefixlen 64 scopeid 0x20
ether da:fd:c4:97:35:c0 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 36 overruns 0 carrier 0 collisions 0
####node2同上操作###
[root@node02 ~]# ifconfig
docker0: flags=4099,BROADCAST,MULTICAST> mtu 1500
inet 172.17.4.1 netmask 255.255.255.0 broadcast 172.17.4.255
ether 02:42:51:7e:94:e3 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.200.120 netmask 255.255.255.0 broadcast 192.168.200.255
inet6 fe80::2ddd:910e:9dd7:3804 prefixlen 64 scopeid 0x20
ether 00:0c:29:49:1a:03 txqueuelen 1000 (Ethernet)
RX packets 222680 bytes 181133492 (172.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 124770 bytes 14292218 (13.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.17.4.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::782c:42ff:feca:531a prefixlen 64 scopeid 0x20
ether 7a:2c:42:ca:53:1a txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 35 overruns 0 carrier 0 collisions 0
# 测试ping通对方docker0网卡 证明flannel起到路由作用
[root@node01 ~]# docker run -it centos:7 /bin/bash
[root@deaaa18a94fa /]# yum install net-tools -y
[root@deaaa18a94fa /]# ifconfig
eth0: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.17.25.2 netmask 255.255.255.0 broadcast 172.17.25.255
[root@deaaa18a94fa /]# ping 172.17.4.1
PING 172.17.4.1 (172.17.4.1) 56(84) bytes of data.
64 bytes from 172.17.4.1: icmp_seq=1 ttl=63 time=0.755 ms
64 bytes from 172.17.4.1: icmp_seq=2 ttl=63 time=2.13 ms
#再次测试ping通两个node中的centos:7容器
[root@node02 ~]# docker run -it centos:7 /bin/bash
[root@6c63efd0712d /]# yum install net-tools -y
[root@6c63efd0712d /]# ifconfig
eth0: flags=4163,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.17.4.2 netmask 255.255.255.0 broadcast 172.17.4.255
[root@6c63efd0712d /]# ping 172.17.25.2
PING 172.17.25.2 (172.17.25.2) 56(84) bytes of data.
64 bytes from 172.17.25.2: icmp_seq=1 ttl=62 time=0.488 ms
64 bytes from 172.17.25.2: icmp_seq=2 ttl=62 time=0.549 ms
6.部署master组件
在master上操作，api-server生成证书
[root@master01 k8s]# ls
etcd-cert etcd-v3.3.10-linux-amd64 flannel-v0.10.0-linux-amd64.tar.gz master.zip
etcd.sh etcd-v3.3.10-linux-amd64.tar.gz kubernetes-server-linux-amd64.tar.gz
[root@master01 k8s]# unzip master.zip
Archive: master.zip
inflating: apiserver.sh
inflating: controller-manager.sh
inflating: scheduler.sh
[root@master01 k8s]# mkdir /opt/kubernetes/{cfg,bin,ssl} -p
[root@master01 k8s]# mkdir k8s-cert
[root@master01 k8s]# cd k8s-cert/
cat > ca-config.json <{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
[root@master01 k8s-cert]# ls
ca-config.json
cat > ca-csr.json <{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
[root@master01 k8s-cert]# ls
ca-config.json ca-csr.json
[root@master01 k8s-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2020/10/08 02:03:35 [INFO] generating a new CA key and certificate from CSR
2020/10/08 02:03:35 [INFO] generate received request
2020/10/08 02:03:35 [INFO] received CSR
2020/10/08 02:03:35 [INFO] generating key: rsa-2048
2020/10/08 02:03:35 [INFO] encoded CSR
2020/10/08 02:03:35 [INFO] signed certificate with serial number 117717371577420932601042163897098685802446401912
[root@master01 k8s-cert]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem
cat > server-csr.json <{
"CN": "kubernetes",
"hosts": [
"10.0.0.1",
"127.0.0.1",
"192.168.200.100", //master1
"192.168.200.90", //master2
"192.168.200.200", //vip
"192.168.200.80", //lb （master）
"192.168.200.70", //lb （backup）
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
[root@master01 k8s-cert]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem server-csr.json
[root@master01 k8s-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
2020/10/08 02:07:24 [INFO] generate received request
2020/10/08 02:07:24 [INFO] received CSR
2020/10/08 02:07:24 [INFO] generating key: rsa-2048
2020/10/08 02:07:24 [INFO] encoded CSR
2020/10/08 02:07:24 [INFO] signed certificate with serial number 430083805945050994239855620117268877861873072530
2020/10/08 02:07:24 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master01 k8s-cert]# ls
ca-config.json ca-csr.json ca.pem server-csr.json server.pem
ca.csr ca-key.pem server.csr server-key.pem
cat > admin-csr.json <{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "system:masters",
"OU": "System"
}
]
}
EOF
[root@master01 k8s-cert]# ls
admin-csr.json ca.csr ca-key.pem server.csr server-key.pem
ca-config.json ca-csr.json ca.pem server-csr.json server.pem
[root@master01 k8s-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
2020/10/08 02:08:16 [INFO] generate received request
2020/10/08 02:08:16 [INFO] received CSR
2020/10/08 02:08:16 [INFO] generating key: rsa-2048
2020/10/08 02:08:16 [INFO] encoded CSR
2020/10/08 02:08:16 [INFO] signed certificate with serial number 128703393960965588319835722368006072668825407118
2020/10/08 02:08:16 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master01 k8s-cert]# ls
admin.csr admin.pem ca-csr.json server.csr server.pem
admin-csr.json ca-config.json ca-key.pem server-csr.json
admin-key.pem ca.csr ca.pem server-key.pem
cat > kube-proxy-csr.json <{
"CN": "system:kube-proxy",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
[root@master01 k8s-cert]# ls
admin.csr admin.pem ca-csr.json kube-proxy-csr.json server-key.pem
admin-csr.json ca-config.json ca-key.pem server.csr server.pem
admin-key.pem ca.csr ca.pem server-csr.json
[root@master01 k8s-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
2020/10/08 02:09:22 [INFO] generate received request
2020/10/08 02:09:22 [INFO] received CSR
2020/10/08 02:09:22 [INFO] generating key: rsa-2048
2020/10/08 02:09:23 [INFO] encoded CSR
2020/10/08 02:09:23 [INFO] signed certificate with serial number 657166277082708480716187649317888036937464272665
2020/10/08 02:09:23 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master01 k8s-cert]# ls
admin.csr ca-config.json ca.pem kube-proxy.pem server.pem
admin-csr.json ca.csr kube-proxy.csr server.csr
admin-key.pem ca-csr.json kube-proxy-csr.json server-csr.json
admin.pem ca-key.pem kube-proxy-key.pem server-key.pem
[root@master01 k8s-cert]# ls *pem
admin-key.pem ca-key.pem kube-proxy-key.pem server-key.pem
admin.pem ca.pem kube-proxy.pem server.pem
[root@master01 k8s-cert]# cp ca*pem server*pem /opt/kubernetes/ssl/
[root@master01 k8s-cert]# cd ..
#解压kubernetes压缩包
[root@master01 k8s]# tar zxvf kubernetes-server-linux-amd64.tar.gz
[root@master01 k8s]# cd kubernetes/server/bin/
#复制关键命令文件
[root@master01 bin]# cp kube-apiserver kubectl kube-controller-manager kube-scheduler /opt/kubernetes/bin/
[root@master01 bin]# cd /root/k8s
#使用 head -c 16 /dev/urandom | od -An -t x | tr -d ' ' 可以随机生成序列号
[root@master01 k8s]# head -c 16 /dev/urandom | od -An -t x | tr -d ' '
2616bd5c1fa27f74c3ddd35d5e9c29f2
[root@master01 k8s]# vim /opt/kubernetes/cfg/token.csv
2616bd5c1fa27f74c3ddd35d5e9c29f2,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
序列号，用户名，id，角色
#二进制文件，token，证书都准备好，开启apiserver
[root@master01 k8s]# bash apiserver.sh 192.168.200.100 https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.
#检查进程是否启动成功
[root@master01 k8s]# ps aux | grep kube
root 69333 37.5 8.3 405152 321744 ? Ssl 02:16 0:12 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 --bind-address=192.168.200.100 --secure-port=6443 --advertise-address=192.168.200.100 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 69348 0.0 0.0 112724 988 pts/1 S+ 02:17 0:00 grep --color=auto kube
#查看配置文件
[root@master01 k8s]# cat /opt/kubernetes/cfg/kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 \
--bind-address=192.168.200.100 \
--secure-port=6443 \
--advertise-address=192.168.200.100 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"
#监听的https端口
[root@master01 k8s]# netstat -ntap | grep 6443
tcp 0 0 192.168.200.100:6443 0.0.0.0:* LISTEN 69333/kube-apiserve
tcp 0 0 192.168.200.100:46264 192.168.200.100:6443 ESTABLISHED 69333/kube-apiserve
tcp 0 0 192.168.200.100:6443 192.168.200.100:46264 ESTABLISHED 69333/kube-apiserve
[root@master01 k8s]# netstat -ntap | grep 8080
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 69333/kube-apiserve
#启动scheduler服务
[root@master01 k8s]# ./scheduler.sh 127.0.0.1
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service.
[root@master01 k8s]# ps aux | grep ku
postfix 69261 0.0 0.1 91732 4084 ? S 02:14 0:00 pickup -l -t unix -u
root 69333 9.3 8.3 405216 322440 ? Ssl 02:16 0:18 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 --bind-address=192.168.200.100 --secure-port=6443 --advertise-address=192.168.200.100 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 69432 2.9 0.5 46128 19488 ? Ssl 02:19 0:00 /opt/kubernetes/bin/kube-scheduler --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect
root 69452 0.0 0.0 112728 984 pts/1 S+ 02:20 0:00 grep --color=auto ku
[root@master01 k8s]# chmod +x controller-manager.sh
#启动controller-manager
[root@master01 k8s]# ./controller-manager.sh 127.0.0.1
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.
[root@master01 k8s]# ps aux | grep kube
root 69333 7.4 8.3 405216 322912 ? Ssl 02:16 0:22 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 --bind-address=192.168.200.100 --secure-port=6443 --advertise-address=192.168.200.100 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 69432 2.4 0.5 47184 20392 ? Ssl 02:19 0:03 /opt/kubernetes/bin/kube-scheduler --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect
root 69517 5.0 1.5 137560 58608 ? Ssl 02:21 0:02 /opt/kubernetes/bin/kube-controller-manager --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect=true --address=127.0.0.1 --service-cluster-ip-range=10.0.0.0/24 --cluster-name=kubernetes --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem --root-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem --experimental-cluster-signing-duration=87600h0m0s
root 69549 0.0 0.0 112724 988 pts/1 S+ 02:22 0:00 grep --color=auto kube
#查看master 节点状态
[root@master01 k8s]# /opt/kubernetes/bin/kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-2 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}
7.node节点部署
//master上操作
//把 kubelet、kube-proxy拷贝到node节点上去
[root@master01 k8s]# cd kubernetes/server/bin/
[root@master01 bin]# scp kubelet kube-proxy [email protected]:/opt/kubernetes/bin/
root@192.168.200.110's password:
kubelet 100% 168MB 72.0MB/s 00:02
kube-proxy 100% 48MB 76.0MB/s 00:00
[root@master01 bin]# scp kubelet kube-proxy [email protected]:/opt/kubernetes/bin/
[email protected]'s password:
kubelet 100% 168MB 101.4MB/s 00:01
kube-proxy 100% 48MB 94.3MB/s 00:00
#nod01节点操作（复制node.zip到/root目录下再解压）
[root@node01 ~]# ls
anaconda-ks.cfg flannel-v0.10.0-linux-amd64.tar.gz node.zip 公共 视频 文档 音乐
flannel.sh initial-setup-ks.cfg README.md 模板 图片 下载 桌面
#解压node.zip，获得kubelet.sh proxy.sh
[root@node01 ~]# unzip node.zip
Archive: node.zip
inflating: proxy.sh
inflating: kubelet.sh
#在master上操作
[root@master01 bin]# cd /root/k8s/
[root@master01 k8s]# mkdir kubeconfig
[root@master01 k8s]# cd kubeconfig/
[root@master01 kubeconfig]# ls
[root@master01 kubeconfig]# rz -E
rz waiting to receive.
[root@master01 kubeconfig]# ls
kubeconfig.sh
[root@master01 kubeconfig]# mv kubeconfig.sh kubeconfig
[root@master01 kubeconfig]# cat /opt/kubernetes/cfg/token.csv
2616bd5c1fa27f74c3ddd35d5e9c29f2,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
[root@master01 kubeconfig]# vim kubeconfig
APISERVER=$1
SSL_DIR=$2
# 创建kubelet bootstrapping kubeconfig
export KUBE_APISERVER="https://$APISERVER:6443"
# 设置集群参数
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=bootstrap.kubeconfig
# 设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap \
--token=2616bd5c1fa27f74c3ddd35d5e9c29f2 \ #修改token信息
--kubeconfig=bootstrap.kubeconfig
# 设置上下文参数
kubectl config set-context default \
--cluster=kubernetes \
--user=kubelet-bootstrap \
--kubeconfig=bootstrap.kubeconfig
# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
#----------------------
# 创建kube-proxy kubeconfig文件
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials kube-proxy \
--client-certificate=$SSL_DIR/kube-proxy.pem \
--client-key=$SSL_DIR/kube-proxy-key.pem \
--embed-certs=true \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kube-proxy \
--kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
#设置环境变量（可以写入到/etc/profile中）
[root@master01 kubeconfig]# vim /etc/profile
export PATH=$PATH:/opt/kubernetes/bin/
[root@master01 kubeconfig]# source /etc/profile
[root@master01 kubeconfig]# kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}
etcd-2 Healthy {"health":"true"}
#生成配置文件
[root@master01 kubeconfig]# bash kubeconfig 192.168.200.100 /root/k8s/k8s-cert/
Cluster "kubernetes" set.
User "kubelet-bootstrap" set.
Context "default" created.
Switched to context "default".
Cluster "kubernetes" set.
User "kube-proxy" set.
Context "default" created.
Switched to context "default".
[root@master01 kubeconfig]# ls
bootstrap.kubeconfig kubeconfig kube-proxy.kubeconfig
#拷贝配置文件到node节点
[root@master01 kubeconfig]# scp bootstrap.kubeconfig kube-proxy.kubeconfig [email protected]:/opt/kubernetes/cfg/
root@192.168.200.110's password:
bootstrap.kubeconfig 100% 2169 1.7MB/s 00:00
kube-proxy.kubeconfig 100% 6275 3.5MB/s 00:00
[root@master01 kubeconfig]# scp bootstrap.kubeconfig kube-proxy.kubeconfig [email protected]:/opt/kubernetes/cfg/
[email protected]'s password:
bootstrap.kubeconfig 100% 2169 1.7MB/s 00:00
kube-proxy.kubeconfig 100% 6275 3.4MB/s 00:00
#创建bootstrap角色赋予权限用于连接apiserver请求签名（关键）
[root@master01 kubeconfig]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created
#在node01节点上操作
[root@node01 ~]# bash kubelet.sh 192.168.200.110
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
#检查kubelet服务启动
[root@node01 ~]# ps aux | grep kube
root 71302 0.1 0.5 325908 22272 ? Ssl 01:40 0:03 /opt/kubernetes/bin/flanneld --ip-masq --etcd-endpoints=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 -etcd-cafile=/opt/etcd/ssl/ca.pem -etcd-certfile=/opt/etcd/ssl/server.pem -etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 76483 1.0 1.1 470876 45012 ? Ssl 02:37 0:00 /opt/kubernetes/bin/kubelet --logtostderr=true --v=4 --hostname-override=192.168.200.110 --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig --bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig --config=/opt/kubernetes/cfg/kubelet.config --cert-dir=/opt/kubernetes/ssl --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0
root 76539 0.0 0.0 112724 988 pts/1 S+ 02:38 0:00 grep --color=auto kube
#master上操作
#检查到node01节点的请求
[root@master01 kubeconfig]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk 87s kubelet-bootstrap Pending（等待集群给该节点颁发证书）
#Master颁发证书
[root@master01 kubeconfig]# kubectl certificate approve node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk
certificatesigningrequest.certificates.k8s.io/node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk approved
#继续查看证书状态
[root@master01 kubeconfig]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk 2m51s kubelet-bootstrap Approved,Issued（已经被允许加入群集）
#查看群集节点，成功加入node01节点
[root@master01 kubeconfig]# kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.200.110 Ready 75s v1.12.3
#在node01节点操作，启动proxy服务
[root@node01 ~]# bash proxy.sh 192.168.200.110
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
[root@node01 ~]# systemctl status kube-proxy.service
● kube-proxy.service - Kubernetes Proxy
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2020-10-08 02:42:16 CST; 15s ago
[root@node01 ~]# systemctl enable kubelet.service
[root@node01 ~]# systemctl enable kube-proxy.service
#node02节点部署
//在node01节点操作
//把现成的/opt/kubernetes目录复制到其他节点进行修改即可
[root@node01 ~]# scp -r /opt/kubernetes/ [email protected]:/opt/
The authenticity of host '192.168.200.120 (192.168.200.120)' can't be established.
ECDSA key fingerprint is SHA256:dMvbIJyuN9aFqJR+OwwLY436gqKEgtipcBLofzOilgU.
ECDSA key fingerprint is MD5:7a:d8:f0:f5:c5:ff:95:36:11:fe:e8:b3:c0:dc:d7:2e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.200.120' (ECDSA) to the list of known hosts.
[email protected]'s password:
flanneld 100% 241 160.0KB/s 00:00
bootstrap.kubeconfig 100% 2169 1.4MB/s 00:00
kube-proxy.kubeconfig 100% 6275 3.0MB/s 00:00
kubelet 100% 379 275.9KB/s 00:00
kubelet.config 100% 269 179.0KB/s 00:00
kubelet.kubeconfig 100% 2298 841.6KB/s 00:00
kube-proxy 100% 191 172.1KB/s 00:00
mk-docker-opts.sh 100% 2139 1.9MB/s 00:00
scp: /opt//kubernetes/bin/flanneld: Text file busy
kubelet 100% 168MB 116.8MB/s 00:01
kube-proxy 100% 48MB 110.6MB/s 00:00
kubelet.crt 100% 2197 760.0KB/s 00:00
kubelet.key 100% 1675 851.2KB/s 00:00
kubelet-client-2020-10-08-02-40-21.pem 100% 1277 612.3KB/s 00:00
kubelet-client-current.pem 100% 1277 825.1KB/s 00:00
#把kubelet，kube-proxy的service文件拷贝到node2中
[root@node01 ~]# scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service [email protected]:/usr/lib/systemd/system/
root@192.168.200.120's password:
kubelet.service 100% 264 262.6KB/s 00:00
kube-proxy.service 100% 231 105.4KB/s 00:00
//在node02上操作，进行修改
//首先删除复制过来的证书，等会node02会自行申请证书
[root@node02 ~]# cd /opt/kubernetes/ssl/
[root@node02 ssl]# ls
kubelet-client-2020-10-08-02-40-21.pem kubelet-client-current.pem kubelet.crt kubelet.key
[root@node02 ssl]# rm -rf *
//修改配置文件kubelet kubelet.config kube-proxy（三个配置文件）
[root@node02 ssl]# cd ../cfg/
[root@node02 cfg]# vim kubelet
KUBELET_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=192.168.200.120 \ # 改成120
--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \
--bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \
--config=/opt/kubernetes/cfg/kubelet.config \
--cert-dir=/opt/kubernetes/ssl \
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"
[root@node02 cfg]# vim kubelet.config
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 192.168.200.120 # 改成120
port: 10250
readOnlyPort: 10255
cgroupDriver: cgroupfs
clusterDNS:
- 10.0.0.2
clusterDomain: cluster.local.
failSwapOn: false
authentication:
anonymous:
enabled: true
[root@node02 cfg]# vim kube-proxy
KUBE_PROXY_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=192.168.200.120 \ #改成120
--cluster-cidr=10.0.0.0/24 \
--proxy-mode=ipvs \
--kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"
//启动服务
[root@node02 cfg]# systemctl start kubelet.service
[root@node02 cfg]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@node02 cfg]# systemctl start kube-proxy.service
[root@node02 cfg]# systemctl enable kube-proxy.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
//在master上操作查看请求
[root@master01 kubeconfig]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk 13m kubelet-bootstrap Approved,Issued
node-csr-lgGd4HsdfVfC56DL3j8U_liiI7ZquBaBfEeAg0OTMkQ 46s kubelet-bootstrap Pending
//授权许可加入群集
[root@master01 kubeconfig]# kubectl certificate approve node-csr-lgGd4HsdfVfC56DL3j8U_liiI7ZquBaBfEeAg0OTMkQ
certificatesigningrequest.certificates.k8s.io/node-csr-lgGd4HsdfVfC56DL3j8U_liiI7ZquBaBfEeAg0OTMkQ approved
[root@master01 kubeconfig]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-BogyYYmDDbmbX0aBxMsx_ETxdt1mPCtSj9FiP3irWsk 14m kubelet-bootstrap Approved,Issued
node-csr-lgGd4HsdfVfC56DL3j8U_liiI7ZquBaBfEeAg0OTMkQ 84s kubelet-bootstrap Approved,Issued
//查看群集中的节点
[root@master01 kubeconfig]# kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.200.110 Ready 12m v1.12.3
192.168.200.120 Ready 44s v1.12.3
单节点部署完毕！
2.多节点部署
在单节点的基础上进行多节点部署
1.master02部署
//优先关闭防火墙和selinux服务
[root@promote ~]# iptables -F
[root@promote ~]# setenforce 0
[root@promote ~]# hostnamectl set-hostname master02
[root@promote ~]# su
//在master01上操作
//复制kubernetes目录到master02
[root@master01 k8s]# scp -r /opt/kubernetes/ [email protected]:/opt
The authenticity of host '192.168.200.90 (192.168.200.90)' can't be established.
ECDSA key fingerprint is SHA256:LGVQSrzmeWOjKsn2nM6C187BdfANy9jsFvmzXotxD7M.
ECDSA key fingerprint is MD5:d2:cd:3a:66:ab:05:b8:16:f8:42:4a:88:4c:60:14:b4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.200.90' (ECDSA) to the list of known hosts.
[email protected]'s password:
token.csv 100% 84 26.7KB/s 00:00
kube-apiserver 100% 939 474.1KB/s 00:00
kube-scheduler 100% 94 37.7KB/s 00:00
kube-controller-manager 100% 483 341.2KB/s 00:00
kube-apiserver 100% 184MB 105.1MB/s 00:01
kubectl 100% 55MB 114.6MB/s 00:00
kube-controller-manager 100% 155MB 115.7MB/s 00:01
kube-scheduler 100% 55MB 103.4MB/s 00:00
ca-key.pem 100% 1679 365.7KB/s 00:00
ca.pem 100% 1359 429.0KB/s 00:00
server-key.pem 100% 1675 1.2MB/s 00:00
server.pem 100% 1643 754.4KB/s 00:00
//复制master中的三个组件启动脚本kube-apiserver.service kube-controller-manager.service kube-scheduler.service
[root@master01 k8s]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service [email protected]:/usr/lib/systemd/system/
root@192.168.200.90's password:
kube-apiserver.service 100% 282 110.1KB/s 00:00
kube-controller-manager.service 100% 317 277.0KB/s 00:00
kube-scheduler.service 100% 281 496.7KB/s 00:00
//master02上操作
//修改配置文件kube-apiserver中的IP
[root@master02 ~]# cd /opt/kubernetes/cfg/
[root@master02 cfg]# vim kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.200.100:2379,https://192.168.200.110:2379,https://192.168.200.120:2379 \
--bind-address=192.168.200.90 \ #改90
--secure-port=6443 \
--advertise-address=192.168.200.90 \ #改90
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"
//特别注意：master02一定要有etcd证书
//需要拷贝master01上已有的etcd证书给master02使用
[root@master01 k8s]# scp -r /opt/etcd/ [email protected]:/opt/
[email protected]'s password:
etcd 100% 523 274.8KB/s 00:00
etcd 100% 18MB 85.9MB/s 00:00
etcdctl 100% 15MB 71.6MB/s 00:00
ca-key.pem 100% 1679 266.4KB/s 00:00
ca.pem 100% 1265 671.4KB/s 00:00
server-key.pem 100% 1679 516.1KB/s 00:00
server.pem 100% 1338 822.6KB/s 00:00
//启动master02中的三个组件服务
[root@master02 cfg]# systemctl start kube-apiserver.service
[root@master02 cfg]# systemctl start kube-controller-manager.service
[root@master02 cfg]# systemctl start kube-scheduler.service
//增加环境变量
[root@master02 cfg]# vim /etc/profile
export PATH=$PATH:/opt/kubernetes/bin/
[root@master02 cfg]# source /etc/profile
[root@master02 cfg]# kubectl get node
NAME STATUS ROLES AGE VERSION
192.168.200.110 Ready 30m v1.12.3
192.168.200.120 Ready 18m v1.12.3
2.lb01配置
[root@localhost ~]# systemctl stop firewalld.service
[root@localhost ~]# setenforce 0
[root@localhost ~]# hostnamectl set-hostname nginx01
[root@localhost ~]# su
[root@nginx01 ~]# vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
[root@nginx01 ~]# yum install nginx -y
//添加四层转发 在events和http中间添加
events {
worker_connections 1024;
}
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
access_log /var/log/nginx/k8s-access.log main;
upstream k8s-apiserver {
server 192.168.200.100:6443;
server 192.168.200.90:6443;
}
server {
listen 6443;
proxy_pass k8s-apiserver;
}
}
http {
[root@nginx01 ~]# systemctl start nginx
//部署keepalived服务
[root@nginx01 ~]# yum install keepalived -y
//修改配置文件
[root@nginx01 ~]# rz -E
rz waiting to receive.
[root@nginx01 ~]# ls
anaconda-ks.cfg keepalived.conf 模板 图片 下载 桌面
initial-setup-ks.cfg 公共 视频 文档 音乐
[root@nginx01 ~]# cp keepalived.conf /etc/keepalived/keepalived.conf
cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes
//注意：lb01是Mster配置如下：
[root@nginx01 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
# 接收邮件地址
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
# 邮件发送地址
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/check_nginx.sh"
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51# VRRP 路由 ID实例，每个实例是唯一的
priority 100 # 优先级，备服务器设置 90
advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.200.200/24 ###
}
track_script {
check_nginx ###
}
}
[root@nginx01 ~]# vim /etc/nginx/check_nginx.sh
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
if [ "$count" -eq 0 ];then
systemctl stop keepalived
fi
[root@nginx01 ~]# chmod +x /etc/nginx/check_nginx.sh
[root@nginx01 ~]# systemctl start keepalived
3.lb2配置
[root@promote ~]# systemctl stop firewalld.service
[root@promote ~]# setenforce 0
[root@promote ~]# hostnamectl set-hostname nginx02
[root@promote ~]# su
[root@nginx02 ~]# vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
[root@nginx02 ~]# yum install nginx -y
//添加四层转发
[root@nginx02 ~]# vim /etc/nginx/nginx.conf
events {
worker_connections 1024;
}
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
access_log /var/log/nginx/k8s-access.log main;
upstream k8s-apiserver {
server 192.168.200.100:6443;
server 192.168.200.90:6443;
}
server {
listen 6443;
proxy_pass k8s-apiserver;
}
}
http {
[root@nginx02 ~]# systemctl restart nginx
//部署keepalived服务
[root@nginx02 ~]# yum install keepalived -y
//修改配置文件
[root@nginx02 ~]# rz -E
rz waiting to receive.
[root@nginx02 ~]# ls
anaconda-ks.cfg keepalived.conf 模板 图片 下载 桌面
initial-setup-ks.cfg 公共 视频 文档 音乐
[root@nginx02 ~]# cp keepalived.conf /etc/keepalived/keepalived.conf
cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes
[root@nginx02 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
# 接收邮件地址
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
# 邮件发送地址
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/nginx/check_nginx.sh"
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.200.200/24
}
track_script {
check_nginx
}
}
[root@nginx02 ~]# vim /etc/nginx/check_nginx.sh
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
if [ "$count" -eq 0 ];then
systemctl stop keepalived
fi
[root@nginx02 ~]# chmod +x /etc/nginx/check_nginx.sh
[root@nginx02 ~]# systemctl start keepalived
4.验证配置
//查看lb01地址信息
[root@nginx01 ~]# ip a
1: lo: ,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:79:9c:93 brd ff:ff:ff:ff:ff:ff
inet 192.168.200.80/24 brd 192.168.200.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 192.168.200.200/24 scope global secondary ens33 //漂移地址在lb01中
valid_lft forever preferred_lft forever
inet6 fe80::b948:c5a0:c6f:e7b7/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: virbr0: -CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 52:54:00:e8:d2:e6 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: ,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:e8:d2:e6 brd ff:ff:ff:ff:ff:ff
//查看lb02地址信息
[root@nginx02 ~]# ip a
1: lo: ,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:05:81:3c brd ff:ff:ff:ff:ff:ff
inet 192.168.200.70/24 brd 192.168.200.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::fc8b:3133:2445:5d32/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: virbr0: -CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 52:54:00:64:ff:7e brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: ,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:64:ff:7e brd ff:ff:ff:ff:ff:ff
//验证地址漂移（lb01中使用pkill nginx，再在lb02中使用ip a 查看）
[root@nginx01 ~]# pkill nginx
[root@nginx01 ~]# ip a
1: lo: ,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:79:9c:93 brd ff:ff:ff:ff:ff:ff
inet 192.168.200.80/24 brd 192.168.200.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::b948:c5a0:c6f:e7b7/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@nginx02 ~]# ip a
1: lo: ,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:05:81:3c brd ff:ff:ff:ff:ff:ff
inet 192.168.200.70/24 brd 192.168.200.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 192.168.200.200/24 scope global secondary ens33 #漂移
//恢复操作（在lb01中先启动nginx服务，再启动keepalived服务）
[root@nginx01 ~]# systemctl start nginx
[root@nginx01 ~]# systemctl start keepalived
[root@nginx01 ~]# ip a
1: lo: ,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: ,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:79:9c:93 brd ff:ff:ff:ff:ff:ff
inet 192.168.200.80/24 brd 192.168.200.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 192.168.200.200/24 scope global secondary ens33 #漂移回来
//nginx站点/usr/share/nginx/html
5.开始修改node节点配置文件统一VIP
//开始修改node节点配置文件统一VIP（bootstrap.kubeconfig,kubelet.kubeconfig）
[root@node02 cfg]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig
server: https://192.168.200.200:6443
[root@node02 cfg]# vim /opt/kubernetes/cfg/kubelet.kubeconfig
server: https://192.168.200.200:6443
[root@node02 cfg]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig
server: https://192.168.200.200:6443
[root@node02 cfg]# systemctl restart kube-proxy.service
[root@node02 cfg]# systemctl restart kubelet.service
#
[root@node01 ~]# cd /opt/kubernetes/cfg/
[root@node01 cfg]# ls
bootstrap.kubeconfig kubelet kubelet.kubeconfig kube-proxy.kubeconfig
flanneld kubelet.config kube-proxy
[root@node01 cfg]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig
[root@node01 cfg]# vim /opt/kubernetes/cfg/kubelet.kubeconfig
[root@node01 cfg]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig
server: https://192.168.200.200:6443
[root@node01 cfg]# systemctl restart kubelet.service
[root@node01 cfg]# systemctl restart kube-proxy.service
替换完成直接自检
[root@node01 cfg]# grep 200.200 *
bootstrap.kubeconfig: server: https://192.168.200.200:6443
kubelet.kubeconfig: server: https://192.168.200.200:6443
kube-proxy.kubeconfig: server: https://192.168.200.200:6443
[root@node02 cfg]# grep 200.200 *
bootstrap.kubeconfig: server: https://192.168.200.200:6443
kubelet.kubeconfig: server: https://192.168.200.200:6443
kube-proxy.kubeconfig: server: https://192.168.200.200:6443
//在lb01上查看nginx的k8s日志
[root@nginx01 ~]# tail /var/log/nginx/k8s-access.log //重启服务产生日志
192.168.200.110 192.168.200.100:6443 - [08/Oct/2020:04:14:23 +0800] 200 1121
192.168.200.110 192.168.200.90:6443 - [08/Oct/2020:04:14:23 +0800] 200 1120
192.168.200.120 192.168.200.90:6443 - [08/Oct/2020:04:14:40 +0800] 200 1120
192.168.200.120 192.168.200.90:6443 - [08/Oct/2020:04:14:40 +0800] 200 1121
6.状态检查
//在master01上操作
//测试创建pod
[root@master01 ~]# kubectl run nginx --image=nginx
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created
//查看状态
[root@master01 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-dbddb74b8-rksr7 1/1 Running 0 39s
注意日志问题，直接查看报错
[root@master01 ~]# kubectl logs nginx-dbddb74b8-rksr7
Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-rksr7)
解决方法 创建匿名用户查看
[root@master01 ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
[root@master01 ~]# kubectl logs nginx-dbddb74b8-rksr7 查看日志
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
//查看pod网络
[root@master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
nginx-dbddb74b8-rksr7 1/1 Running 0 3m2s 172.17.25.2 192.168.200.110
//在对应网段的node节点上操作可以直接访问
[root@node01 cfg]# curl 172.17.25.2
<!DOCTYPE html>
Welcome to nginx<span class="token operator">!</span><<span class="token operator">/</span>title>
<style>
body <span class="token punctuation">{</span>
width: 35em<span class="token punctuation">;</span>
margin: 0 auto<span class="token punctuation">;</span>
font<span class="token operator">-</span>family: Tahoma<span class="token punctuation">,</span> Verdana<span class="token punctuation">,</span> Arial<span class="token punctuation">,</span> sans<span class="token operator">-</span>serif<span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<<span class="token operator">/</span>style>
<<span class="token operator">/</span>head>
<body>
<h1>Welcome to nginx<span class="token operator">!</span><<span class="token operator">/</span>h1>
<p><span class="token keyword">If</span> you see this page<span class="token punctuation">,</span> the nginx web server is successfully installed and
working<span class="token punctuation">.</span> Further configuration is required<span class="token punctuation">.</span><<span class="token operator">/</span>p>
<p><span class="token keyword">For</span> online documentation and support please refer to
<a href=<span class="token string">"http://nginx.org/"</span>>nginx<span class="token punctuation">.</span>org<<span class="token operator">/</span>a><span class="token punctuation">.</span><br<span class="token operator">/</span>>
Commercial support is available at
<a href=<span class="token string">"http://nginx.com/"</span>>nginx<span class="token punctuation">.</span>com<<span class="token operator">/</span>a><span class="token punctuation">.</span><<span class="token operator">/</span>p>
<p><em>Thank you <span class="token keyword">for</span> <span class="token keyword">using</span> nginx<span class="token punctuation">.</span><<span class="token operator">/</span>em><<span class="token operator">/</span>p>
<<span class="token operator">/</span>body>
<<span class="token operator">/</span>html>
<span class="token operator">/</span><span class="token operator">/</span>访问就会产生日志
<span class="token operator">/</span><span class="token operator">/</span>回到master01操作
<span class="token namespace">[root@master01 ~]</span><span class="token comment"># kubectl logs nginx-dbddb74b8-rksr7</span>
<span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>sh: <span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>d<span class="token operator">/</span> is not empty<span class="token punctuation">,</span> will attempt to perform configuration
<span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>sh: Looking <span class="token keyword">for</span> shell scripts in <span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>d<span class="token operator">/</span>
<span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>sh: Launching <span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>d<span class="token operator">/</span>10<span class="token operator">-</span>listen<span class="token operator">-</span>on<span class="token operator">-</span>ipv6<span class="token operator">-</span>by<span class="token operator">-</span>default<span class="token punctuation">.</span>sh
10<span class="token operator">-</span>listen<span class="token operator">-</span>on<span class="token operator">-</span>ipv6<span class="token operator">-</span>by<span class="token operator">-</span>default<span class="token punctuation">.</span>sh: Getting the checksum of <span class="token operator">/</span>etc<span class="token operator">/</span>nginx<span class="token operator">/</span>conf<span class="token punctuation">.</span>d<span class="token operator">/</span>default<span class="token punctuation">.</span>conf
10<span class="token operator">-</span>listen<span class="token operator">-</span>on<span class="token operator">-</span>ipv6<span class="token operator">-</span>by<span class="token operator">-</span>default<span class="token punctuation">.</span>sh: Enabled listen on IPv6 in <span class="token operator">/</span>etc<span class="token operator">/</span>nginx<span class="token operator">/</span>conf<span class="token punctuation">.</span>d<span class="token operator">/</span>default<span class="token punctuation">.</span>conf
<span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>sh: Launching <span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>d<span class="token operator">/</span>20<span class="token operator">-</span>envsubst<span class="token operator">-</span>on<span class="token operator">-</span>templates<span class="token punctuation">.</span>sh
<span class="token operator">/</span>docker<span class="token operator">-</span>entrypoint<span class="token punctuation">.</span>sh: Configuration complete<span class="token punctuation">;</span> ready <span class="token keyword">for</span> <span class="token function">start</span> up
172<span class="token punctuation">.</span>17<span class="token punctuation">.</span>25<span class="token punctuation">.</span>1 <span class="token operator">-</span> <span class="token operator">-</span> <span class="token punctuation">[</span>07<span class="token operator">/</span>Oct<span class="token operator">/</span>2020:20:21:12 <span class="token operator">+</span>0000<span class="token punctuation">]</span> <span class="token string">"GET / HTTP/1.1"</span> 200 612 <span class="token string">"-"</span> <span class="token string">"curl/7.29.0"</span> <span class="token string">"-"</span>
</code></pre>
</div>
</div>
</div>
</div>
</div>
<!--PC和WAP自适应版-->
<div id="SOHUCS" sid="1560860774807703552"></div>
<script type="text/javascript" src="/views/front/js/chanyan.js"></script>
<!-- 文章页-底部 动态广告位 -->
<div class="youdao-fixed-ad" id="detail_ad_bottom"></div>
</div>
<div class="col-md-3">
<div class="row" id="ad">
<!-- 文章页-右侧1 动态广告位 -->
<div id="right-1" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_1"> </div>
</div>
<!-- 文章页-右侧2 动态广告位 -->
<div id="right-2" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_2"></div>
</div>
<!-- 文章页-右侧3 动态广告位 -->
<div id="right-3" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_3"></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="container">
<h4 class="pt20 mb15 mt0 border-top">你可能感兴趣的:(kubernetes,kubernetes,k8s多节点二进制部署,k8s单节点二进制部署,k8s集群,云计算)</h4>
<div id="paradigm-article-related">
<div class="recommend-post mb30">
<ul class="widget-links">
<li><a href="/article/1950227023192453120.htm"
title="MotionLCM 部署优化 踩坑解决bug" target="_blank">MotionLCM 部署优化 踩坑解决bug</a>
<span class="text-muted">AI算法网奇</span>
<a class="tag" taget="_blank" href="/search/aigc%E4%B8%8E%E6%95%B0%E5%AD%97%E4%BA%BA/1.htm">aigc与数字人</a><a class="tag" taget="_blank" href="/search/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E5%AE%9D%E5%85%B8/1.htm">深度学习宝典</a><a class="tag" taget="_blank" href="/search/%E6%96%87%E7%94%9Fmotion/1.htm">文生motion</a>
<div>目录依赖项windowstorchok：渲染黑白图问题解决：humanml3d：sentence-t5-large下载数据：报错：Nomodulenamed'sentence_transformers'继续报错：fromtransformers.integrationsimportCodeCarbonCallback解决方法：推理相关转mesh：module‘matplotlib.cm‘hasno</div>
</li>
<li><a href="/article/1950225255079407616.htm"
title="企业级区块链平台Hyperchain核心原理剖析" target="_blank">企业级区块链平台Hyperchain核心原理剖析</a>
<span class="text-muted">boyedu</span>
<a class="tag" taget="_blank" href="/search/%E5%8C%BA%E5%9D%97%E9%93%BE/1.htm">区块链</a><a class="tag" taget="_blank" href="/search/%E5%8C%BA%E5%9D%97%E9%93%BE/1.htm">区块链</a><a class="tag" taget="_blank" href="/search/%E4%BC%81%E4%B8%9A%E7%BA%A7%E5%8C%BA%E5%9D%97%E9%93%BE%E5%B9%B3%E5%8F%B0/1.htm">企业级区块链平台</a><a class="tag" taget="_blank" href="/search/Hyperchain/1.htm">Hyperchain</a>
<div>Hyperchain作为国产自主可控的企业级联盟区块链平台，其核心原理围绕高性能共识、隐私保护、智能合约引擎及可扩展架构展开，通过多模块协同实现企业级区块链网络的高效部署与安全运行。以下从核心架构、关键技术、性能优化、安全机制、应用场景五个维度展开剖析：一、核心架构：分层解耦与模块化设计Hyperchain采用分层架构，将区块链功能解耦为独立模块，支持灵活组合与扩展：P2P网络层由验证节点（VP）</div>
</li>
<li><a href="/article/1950207853105049600.htm"
title="【Coze搞钱实战】3. 避坑指南：对话流设计中的6个致命错误（真实案例）" target="_blank">【Coze搞钱实战】3. 避坑指南：对话流设计中的6个致命错误（真实案例）</a>
<span class="text-muted">AI_DL_CODE</span>
<a class="tag" taget="_blank" href="/search/Coze%E5%B9%B3%E5%8F%B0/1.htm">Coze平台</a><a class="tag" taget="_blank" href="/search/%E5%AF%B9%E8%AF%9D%E6%B5%81%E8%AE%BE%E8%AE%A1/1.htm">对话流设计</a><a class="tag" taget="_blank" href="/search/%E5%AE%A2%E6%9C%8DBot%E9%81%BF%E5%9D%91/1.htm">客服Bot避坑</a><a class="tag" taget="_blank" href="/search/%E7%94%A8%E6%88%B7%E6%B5%81%E5%A4%B1/1.htm">用户流失</a><a class="tag" taget="_blank" href="/search/%E5%B0%81%E5%8F%B7%E9%A3%8E%E9%99%A9/1.htm">封号风险</a><a class="tag" taget="_blank" href="/search/%E6%99%BA%E8%83%BD%E5%AE%A2%E6%9C%8D%E9%85%8D%E7%BD%AE/1.htm">智能客服配置</a><a class="tag" taget="_blank" href="/search/%E6%95%85%E9%9A%9C%E4%BF%AE%E5%A4%8D%E6%8C%87%E5%8D%97/1.htm">故障修复指南</a>
<div>摘要：对话流设计是智能客服Bot能否落地的核心环节，直接影响用户体验与业务安全。本文基于50+企业Bot部署故障分析，聚焦导致用户流失、投诉甚至封号的6大致命错误：无限循环追问、人工移交超时、敏感词过滤缺失、知识库冲突、未处理否定意图、跨平台适配失败。通过真实案例拆解每个错误的表现形式、技术根因及工业级解决方案，提供可直接复用的Coze配置代码、工作流模板和检测工具。文中包含对话流健康度检测工具使</div>
</li>
<li><a href="/article/1950205081676738560.htm"
title="K8s常用的命令" target="_blank">K8s常用的命令</a>
<span class="text-muted">尚未来-</span>
<a class="tag" taget="_blank" href="/search/%E8%BF%90%E7%BB%B4/1.htm">运维</a><a class="tag" taget="_blank" href="/search/k8s/1.htm">k8s</a>
<div>一、基础命令查看集群信息bashkubectlcluster-info#显示集群端点和服务信息查看节点bashkubectlgetnodes#列出所有节点kubectldescribenode#查看节点详细信息查看命名空间bashkubectlgetnamespaces#列出所有命名空间切换命名空间bashkubectlconfigset-context--current--namespace=二</div>
</li>
<li><a href="/article/1950202936449626112.htm"
title="Qwen3 大模型实战：使用 vLLM 部署与函数调用（Function Call）全攻略" target="_blank">Qwen3 大模型实战：使用 vLLM 部署与函数调用（Function Call）全攻略</a>
<span class="text-muted">曦紫沐</span>
<a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B/1.htm">大模型</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2/1.htm">大模型部署</a><a class="tag" taget="_blank" href="/search/Qwen3/1.htm">Qwen3</a><a class="tag" taget="_blank" href="/search/vLLM/1.htm">vLLM</a><a class="tag" taget="_blank" href="/search/%E5%87%BD%E6%95%B0%E8%B0%83%E7%94%A8/1.htm">函数调用</a>
<div>文章摘要本文将带你从零开始，深入掌握如何使用Qwen3-8B大语言模型，结合vLLM进行高性能部署，并通过函数调用（FunctionCall）实现模型与外部工具的智能联动。我们将详细讲解部署命令、调用方式、代码示例及实际应用场景，帮助你快速构建基于Qwen3的智能应用。一、Qwen3简介与部署环境准备Qwen3是通义千问系列的最新一代大语言模型，具备强大的自然语言理解和生成能力，尤其在函数调用、工</div>
</li>
<li><a href="/article/1950202938941042688.htm"
title="大模型量化终极对决：FP8 vs AWQ INT4，谁才是性能与精度的王者？" target="_blank">大模型量化终极对决：FP8 vs AWQ INT4，谁才是性能与精度的王者？</a>
<span class="text-muted">曦紫沐</span>
<a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B/1.htm">大模型</a><a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E9%87%8F%E5%8C%96/1.htm">大模型量化</a><a class="tag" taget="_blank" href="/search/FP8/1.htm">FP8</a><a class="tag" taget="_blank" href="/search/AWQ_INT4/1.htm">AWQ_INT4</a>
<div>摘要在大模型部署与优化中，量化技术是突破性能瓶颈的关键。FP8量化与AWQINT4量化作为当前主流方案，分别以“高精度”和“极致压缩”为核心优势。本文通过表格对比二者的数据格式、精度损失、硬件依赖及适用场景，助您在不同需求下精准选择最优方案。一、数据格式：浮点与整数的底层差异FP8量化采用浮点数（FP8），包含E4M3（4位阶码+3位尾数）和E5M2（5位阶码+2位尾数）两种格式，保留动态范围；而</div>
</li>
<li><a href="/article/1950201927899869184.htm"
title="vllm本地台式机运行(3070显存8G)" target="_blank">vllm本地台式机运行(3070显存8G)</a>
<span class="text-muted">名明鸣冥</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/Qwen/1.htm">Qwen</a><a class="tag" taget="_blank" href="/search/vllm/1.htm">vllm</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2/1.htm">大模型部署</a>
<div>ollama和vllm的对比这块对比网上都很多资料了，这边使用上简单感觉就是ollama很方便部署,占用资料更少,但后续性能会差点,各选项也不一样.ollama安装和使用和docker很像,就不赘述了,这里是看vllm的安装硬件情况介绍30708G显存+-------------------------------------------------------------------------</div>
</li>
<li><a href="/article/1950197640729784320.htm"
title="从振动信号到精准预警：AI 如何重塑工业设备健康管理？" target="_blank">从振动信号到精准预警：AI 如何重塑工业设备健康管理？</a>
<span class="text-muted">缘华工业智维</span>
<a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a><a class="tag" taget="_blank" href="/search/%E8%AE%A1%E7%AE%97%E6%9C%BA%E8%A7%86%E8%A7%89/1.htm">计算机视觉</a><a class="tag" taget="_blank" href="/search/%E8%BE%B9%E7%BC%98%E8%AE%A1%E7%AE%97/1.htm">边缘计算</a><a class="tag" taget="_blank" href="/search/%E4%BF%A1%E6%81%AF%E4%B8%8E%E9%80%9A%E4%BF%A1/1.htm">信息与通信</a>
<div>在智能制造浪潮席卷全球的当下，工业生产正经历着从传统模式向智能化、数字化转型的深刻变革。在这场变革中，AI驱动的振动分析技术犹如一颗璀璨新星，成为工业设备可靠运行的“健康卫士”。它通过在设备关键部位部署振动传感器，如同医生为患者听诊般实时采集设备运行时的振动信号，再借助强大的人工智能算法对这些“工业脉搏”进行深度解析，从而实现对工业设备从故障预警到寿命预测的全周期精准守护。一、AI振动分析：设备状</div>
</li>
<li><a href="/article/1950193733681082368.htm"
title="基于Python的AI健康助手：开发与部署全攻略" target="_blank">基于Python的AI健康助手：开发与部署全攻略</a>
<span class="text-muted">AI算力网络与通信</span>
<a class="tag" taget="_blank" href="/search/AI%E7%AE%97%E5%8A%9B%E7%BD%91%E7%BB%9C%E4%B8%8E%E9%80%9A%E4%BF%A1%E5%8E%9F%E7%90%86/1.htm">AI算力网络与通信原理</a><a class="tag" taget="_blank" href="/search/AI%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD%E5%A4%A7%E6%95%B0%E6%8D%AE%E6%9E%B6%E6%9E%84/1.htm">AI人工智能大数据架构</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a><a class="tag" taget="_blank" href="/search/ai/1.htm">ai</a>
<div>基于Python的AI健康助手：开发与部署全攻略关键词：Python、AI健康助手、机器学习、自然语言处理、Flask、部署、健康管理摘要：本文将详细介绍如何使用Python开发一个AI健康助手，从需求分析、技术选型到核心功能实现，再到最终部署上线的完整过程。我们将使用自然语言处理技术理解用户健康咨询，通过机器学习模型提供个性化建议，并展示如何用Flask框架构建Web应用接口。文章包含大量实际代</div>
</li>
<li><a href="/article/1950192091439099904.htm"
title="K8S 常用命令全解析：高效管理容器化集群" target="_blank">K8S 常用命令全解析：高效管理容器化集群</a>
<span class="text-muted">恩爸编程</span>
<a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/kubernetes/1.htm">kubernetes</a><a class="tag" taget="_blank" href="/search/%E5%AE%B9%E5%99%A8/1.htm">容器</a><a class="tag" taget="_blank" href="/search/k8s%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/1.htm">k8s常用命令</a><a class="tag" taget="_blank" href="/search/k8s%E6%9C%89%E5%93%AA%E4%BA%9B%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/1.htm">k8s有哪些常用命令</a><a class="tag" taget="_blank" href="/search/k8s%E5%91%BD%E4%BB%A4%E6%9C%89%E5%93%AA%E4%BA%9B/1.htm">k8s命令有哪些</a><a class="tag" taget="_blank" href="/search/K8S%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4%E6%9C%89%E5%93%AA%E4%BA%9B/1.htm">K8S常用命令有哪些</a>
<div>K8S常用命令全解析：高效管理容器化集群一、引言Kubernetes（K8S）作为强大的容器编排平台，其丰富的命令行工具（kubectl）为用户提供了便捷的方式来管理集群中的各种资源。熟练掌握K8S常用命令对于开发人员和运维人员至关重要，能够有效提高容器化应用的部署、监控与维护效率。本文将详细介绍一些K8S常用命令及其使用案例。二、基础资源操作命令（一）kubectlcreate功能：用于创建K8</div>
</li>
<li><a href="/article/1950190073274232832.htm"
title="k8s常用基础命令总结" target="_blank">k8s常用基础命令总结</a>
<span class="text-muted">Tony666688888</span>
<a class="tag" taget="_blank" href="/search/kubernetes/1.htm">kubernetes</a><a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/%E5%AE%B9%E5%99%A8/1.htm">容器</a><a class="tag" taget="_blank" href="/search/k8s/1.htm">k8s</a>
<div>----------------------k8s常用基础命令---------------------------------获取Pod信息#1.获取k8s的命名空间kubectlgetnamespaces1)获取Pod列表及简要信息：kubectlgetpods2)以YAML格式获取Pod详细信息：kubectlgetpod-oyaml3)获取特定命名空间中的Pod列表kubectlgetpo</div>
</li>
<li><a href="/article/1950186798126788608.htm"
title="【三桥君】MCP中台，究竟如何实现多模型、多渠道、多环境的统一管控？如何以MCP为核心设计AI应用架构？" target="_blank">【三桥君】MCP中台，究竟如何实现多模型、多渠道、多环境的统一管控？如何以MCP为核心设计AI应用架构？</a>
<span class="text-muted">三桥君</span>
<a class="tag" taget="_blank" href="/search/%E3%80%8A%E4%B8%89%E6%A1%A5%E5%90%9B/1.htm">《三桥君</a><a class="tag" taget="_blank" href="/search/MCP%E8%90%BD%E5%9C%B0%E6%96%B9%E6%B3%95%E8%AE%BA%E3%80%8B/1.htm">MCP落地方法论》</a><a class="tag" taget="_blank" href="/search/%E3%80%8A%E4%B8%89%E6%A1%A5%E5%90%9B/1.htm">《三桥君</a><a class="tag" taget="_blank" href="/search/AI%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%90%BD%E5%9C%B0%E6%96%B9%E6%B3%95%E8%AE%BA%E3%80%8B/1.htm">AI大模型落地方法论》</a><a class="tag" taget="_blank" href="/search/%23/1.htm">#</a><a class="tag" taget="_blank" href="/search/%E3%80%8A%E4%B8%89%E6%A1%A5%E5%90%9B/1.htm">《三桥君</a><a class="tag" taget="_blank" href="/search/AI%E4%BA%A7%E5%93%81%E6%96%B9%E6%B3%95%E8%AE%BA%E3%80%8B/1.htm">AI产品方法论》</a><a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a><a class="tag" taget="_blank" href="/search/AI%E4%BA%A7%E5%93%81%E7%BB%8F%E7%90%86/1.htm">AI产品经理</a><a class="tag" taget="_blank" href="/search/MCP/1.htm">MCP</a><a class="tag" taget="_blank" href="/search/API/1.htm">API</a><a class="tag" taget="_blank" href="/search/%E4%B8%89%E6%A1%A5%E5%90%9B/1.htm">三桥君</a><a class="tag" taget="_blank" href="/search/%E7%B3%BB%E7%BB%9F%E6%9E%B6%E6%9E%84/1.htm">系统架构</a><a class="tag" taget="_blank" href="/search/llama/1.htm">llama</a>
<div>你好，我是✨三桥君✨本文介绍>>一、引言随着人工智能技术的快速发展，越来越多的企业开始引入大语言模型（LLM）以提升用户体验和运营效率。然而，如何高效、稳定地将这些AI能力落地到生产环境呢？传统的系统架构往往难以应对AI应用的高并发、低延迟和灵活扩展需求，因此，从整体架构角度设计AI应用架构显得尤为重要。本文三桥君将深入探讨以MCP为核心的AI应用架构，并分析多种部署方式的优劣势，为企业在AI落地</div>
</li>
<li><a href="/article/1950185789447008256.htm"
title="Python 程序设计讲义（26）：字符串的用法——字符的编码" target="_blank">Python 程序设计讲义（26）：字符串的用法——字符的编码</a>
<span class="text-muted">睿思达DBA_WGX</span>
<a class="tag" taget="_blank" href="/search/Python/1.htm">Python</a><a class="tag" taget="_blank" href="/search/%E8%AE%B2%E4%B9%89/1.htm">讲义</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>Python程序设计讲义（26）：字符串的用法——字符的编码目录Python程序设计讲义（26）：字符串的用法——字符的编码一、字符的编码二、`ASCII`编码三、`Unicode`编码四、使用`ord()`函数查询一个字符对应的`Unicode`编码五、使用`chr()`函数查询一个`Unicode`编码对应的字符六、`Python`字符串的特征一、字符的编码计算机默认只能处理二进制数，而不能处</div>
</li>
<li><a href="/article/1950185537319006208.htm"
title="新手如何通过github pages静态网站托管搭建个人网站和项目站点" target="_blank">新手如何通过github pages静态网站托管搭建个人网站和项目站点</a>
<span class="text-muted">vvandre</span>
<a class="tag" taget="_blank" href="/search/Web%E6%8A%80%E6%9C%AF/1.htm">Web技术</a><a class="tag" taget="_blank" href="/search/github/1.htm">github</a>
<div>一、githubpages静态网站托管介绍githubpages它是一个免费快捷的静态网站托管服务。对比传统建站，它有哪些优点呢？在传统方式中，首先要租用服务器，服务器上需要运行外部程序，还需要再购买域名，要配置SSL证书，最后还要配置DNS，将域名解析到服务器。这一套繁琐操作，基本上就把小白劝退了。graphTDA[租用服务器]-->B[部署Web应用(运行外部程序，如Nginx)]B-->C[</div>
</li>
<li><a href="/article/1950176839670493184.htm"
title="全局修改GitLab14默认语言为中文" target="_blank">全局修改GitLab14默认语言为中文</a>
<span class="text-muted"></span>
<div>GitLab安装成功后默认语言是英语，只有登录后才能手动指定为中文，且这个配置只对自己生效，经查阅资料后，总结全局修改GitLab14默认语言为中文方法如下：0.进入容器如果你用Docker部署的GitLab，那么需要使用命令sudodockerexec-itgitlab/bin/bash进入容器1.修改rails配置文件打开/opt/gitlab/embedded/service/gitlab-</div>
</li>
<li><a href="/article/1950175452580605952.htm"
title="Gerapy爬虫管理框架深度解析：企业级分布式爬虫管控平台" target="_blank">Gerapy爬虫管理框架深度解析：企业级分布式爬虫管控平台</a>
<span class="text-muted">Python×CATIA工业智造</span>
<a class="tag" taget="_blank" href="/search/%E7%88%AC%E8%99%AB/1.htm">爬虫</a><a class="tag" taget="_blank" href="/search/%E5%88%86%E5%B8%83%E5%BC%8F/1.htm">分布式</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/pycharm/1.htm">pycharm</a>
<div>引言：爬虫工程化的必然选择随着企业数据采集需求指数级增长，传统单点爬虫管理模式面临三重困境：管理效率瓶颈：手动部署耗时占开发总时长的40%以上系统可靠性低：研究显示超过65%的爬虫故障源于部署或调度错误资源利用率差：平均爬虫服务器CPU利用率不足30%爬虫管理方案对比：┌───────────────┬─────────────┬───────────┬───────────┬──────────</div>
</li>
<li><a href="/article/1950169525244719104.htm"
title="从零到一：基于差分隐私决策树的客户购买预测系统实战开发" target="_blank">从零到一：基于差分隐私决策树的客户购买预测系统实战开发</a>
<span class="text-muted">笙囧同学</span>
<a class="tag" taget="_blank" href="/search/%E5%86%B3%E7%AD%96%E6%A0%91/1.htm">决策树</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/1.htm">机器学习</a>
<div>作者简介：笙囧同学，中科院计算机大模型方向硕士，全栈开发爱好者联系方式：3251736703@qq.com各大平台账号：笙囧同学座右铭：偷懒是人生进步的阶梯文章导航快速导航前言-项目背景与价值项目概览-系统架构与功能技术深度解析-核心算法原理️系统实现详解-工程实践细节性能评估与分析-实验结果分析Web系统开发-前后端开发部署与运维-DevOps实践完整复现指南-手把手教程️实践案例与故障排除-问</div>
</li>
<li><a href="/article/1950169145177862144.htm"
title="如何运用深度学习打造高效AI人工智能系统" target="_blank">如何运用深度学习打造高效AI人工智能系统</a>
<span class="text-muted">AI智能探索者</span>
<a class="tag" taget="_blank" href="/search/AI/1.htm">AI</a><a class="tag" taget="_blank" href="/search/Agent/1.htm">Agent</a><a class="tag" taget="_blank" href="/search/%E6%99%BA%E8%83%BD%E4%BD%93%E5%BC%80%E5%8F%91%E5%AE%9E%E6%88%98/1.htm">智能体开发实战</a><a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a><a class="tag" taget="_blank" href="/search/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/1.htm">深度学习</a><a class="tag" taget="_blank" href="/search/ai/1.htm">ai</a>
<div>如何运用深度学习打造高效AI人工智能系统关键词：深度学习、AI系统、神经网络、模型优化、实战开发摘要：本文将从深度学习的核心概念出发，结合生活实例和代码实战，系统讲解如何构建高效AI系统。我们会拆解数据准备、模型设计、训练优化、部署落地的全流程，揭秘“数据-模型-训练-推理”的协同机制，并通过具体案例演示从0到1开发AI系统的关键技巧，帮助开发者掌握打造高效AI系统的底层逻辑。背景介绍目的和范围在</div>
</li>
<li><a href="/article/1950161707464716288.htm"
title="Ubuntu Docker 安装Redis" target="_blank">Ubuntu Docker 安装Redis</a>
<span class="text-muted">LLLL96</span>
<a class="tag" taget="_blank" href="/search/Ubuntu/1.htm">Ubuntu</a><a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/redis/1.htm">redis</a><a class="tag" taget="_blank" href="/search/ubuntu/1.htm">ubuntu</a>
<div>目录介绍1.数据结构丰富2.高性能3.持久化1.拉取Redis镜像2.创建挂载目录(可选)3.配置Redis持久化(可选)4.使用配置文件运行容器5.查看redis日志介绍1.数据结构丰富Redis支持多种数据结构，包括：字符串（String）:可以用来存储任何类型的数据，例如文本、数字或二进制数据。哈希（Hash）:存储字段和值的映射，适合用于表示对象。列表（List）:有序的字符串列表，可以用</div>
</li>
<li><a href="/article/1950159185115148288.htm"
title="位运算 原码、补码" target="_blank">位运算 原码、补码</a>
<span class="text-muted">YouQian772</span>
<a class="tag" taget="_blank" href="/search/%E7%AC%94%E8%AE%B0/1.htm">笔记</a><a class="tag" taget="_blank" href="/search/%E4%BD%8D%E8%BF%90%E7%AE%97/1.htm">位运算</a><a class="tag" taget="_blank" href="/search/c%2B%2B/1.htm">c++</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a>
<div>1.位运算//按位与：对应位都为1才为1a&b//按位或：对应位有一个为1就为1a|b//按位异或：对应位不同则为1a^b//按位取反：0变1，1变0（注意符号位）~a//左移：a>n相当于a/2^n（向下取整）b>>1a=10→二进制00001010（8位表示，符号位为0）a>>2→向右移2位，右侧2位丢弃，左侧补0→00000010→结果为2a=-10→原码10001010→补码1111011</div>
</li>
<li><a href="/article/1950144063588069376.htm"
title="Serverless架构下Spring Function的创新实践" target="_blank">Serverless架构下Spring Function的创新实践</a>
<span class="text-muted">tmjpz04412</span>
<a class="tag" taget="_blank" href="/search/serverless/1.htm">serverless</a><a class="tag" taget="_blank" href="/search/%E6%9E%B6%E6%9E%84/1.htm">架构</a><a class="tag" taget="_blank" href="/search/spring/1.htm">spring</a>
<div>引言：Serverless与Spring生态的交汇背景介绍：云计算与Serverless架构的兴起Spring生态的演进与云原生适配性核心问题：传统Spring应用如何融入Serverless范式Serverless架构的核心特征与挑战事件驱动、弹性伸缩与按需计费冷启动问题与性能优化需求Spring应用在Serverless环境中的典型瓶颈（如依赖注入、上下文初始化）SpringFunction的</div>
</li>
<li><a href="/article/1950139138351099904.htm"
title="Java中什么是类加载？类加载的过程？" target="_blank">Java中什么是类加载？类加载的过程？</a>
<span class="text-muted"></span>
<div>类加载指的是把类加载到JVM中。把二进制流存储到内存中，之后经过一番解析、处理转化成可用的class类二进制流可以来源于class文件，或通过字节码工具生成的字节码或来自于网络。只要符合格式的二进制流，JVM来者不拒。虚拟机遇到⼀条new指令时，⾸先将去检查这个指令的参数是否能在常量池中定位到这个类的符号引⽤，并且检查这个符号引⽤代表的类是否已被加载过、解析和初始化过。如果没有，那必须先执⾏相应的</div>
</li>
<li><a href="/article/1950137877904027648.htm"
title="Docker部署Minio" target="_blank">Docker部署Minio</a>
<span class="text-muted">YiShuoChen666</span>
<a class="tag" taget="_blank" href="/search/eureka/1.htm">eureka</a><a class="tag" taget="_blank" href="/search/%E4%BA%91%E5%8E%9F%E7%94%9F/1.htm">云原生</a><a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/minio/1.htm">minio</a>
<div>一、拉取镜像dockerpullminio/minio:RELEASE.2025-04-22T22-12-26Z注：这里使用的版本是RELEASE.2025-04-22T22-12-26Z，最新的版本Web-UI界面没有管理bucket和其它的操作按钮，需要通过mc命令来实现，这对于新手很不友好二、创建目录，来存放minio的配置和将来放的文件mkdir-p/home/minio/configmk</div>
</li>
<li><a href="/article/1950131321980383232.htm"
title="深入了解 Kubernetes（k8s）：从概念到实践" target="_blank">深入了解 Kubernetes（k8s）：从概念到实践</a>
<span class="text-muted"></span>
<div>目录一、k8s核心概念二、k8s的优势三、k8s架构组件控制平面组件节点组件四、k8s+docker运行前后端分离项目的例子1.准备前端项目2.准备后端项目3.创建k8s部署配置文件4.部署应用到k8s集群在当今云计算和容器化技术飞速发展的时代，Kubernetes（简称k8s）已成为容器编排领域的事实标准。无论是互联网巨头、传统企业还是初创公司，都在广泛采用k8s来管理和部署容器化应用。本文将带</div>
</li>
<li><a href="/article/1950131322404007936.htm"
title="【计算机网络】细说IP" target="_blank">【计算机网络】细说IP</a>
<span class="text-muted">问道飞鱼</span>
<a class="tag" taget="_blank" href="/search/%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BD%91%E7%BB%9C/1.htm">计算机网络</a><a class="tag" taget="_blank" href="/search/%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BD%91%E7%BB%9C/1.htm">计算机网络</a><a class="tag" taget="_blank" href="/search/tcp%2Fip/1.htm">tcp/ip</a><a class="tag" taget="_blank" href="/search/%E7%BD%91%E7%BB%9C%E5%8D%8F%E8%AE%AE/1.htm">网络协议</a>
<div>文章目录概述IP地址的组成IP地址的分类IP地址的作用分类一、A类IP地址二、B类IP地址三、C类IP地址四、D类IP地址五、E类IP地址协议报文子网掩码一、定义与功能二、表示方法三、子网掩码与IP地址的关系四、子网掩码的设置与配置五、实例说明IPv6一、定义与背景二、地址格式与特点三、优势与功能四、过渡与部署五、应用与发展IPv6协议报文有了IPv6还需要子网掩码吗概述IP，全称Internet</div>
</li>
<li><a href="/article/1950119980049690624.htm"
title="云服务器如何搭建多站点？Nginx多域名部署方案详解 (2025)" target="_blank">云服务器如何搭建多站点？Nginx多域名部署方案详解 (2025)</a>
<span class="text-muted"></span>
<div>更多云服务器知识，尽在hostol.com当你拥有了第一台云服务器时，那种感觉，就如同得到了一块充满无限可能的“数字画布”。很多人的第一反应是，将自己最心爱的那个域名和网站部署上去，看着它在互联网上成功“点亮”。但是，随着你的项目增多、想法迸发，你可能会开始思考一个极具性价比的问题：我能在这同一台服务器上，再多放几个网站吗？比如，除了我的主博客之外，再放一个作品集网站，一个用于测试的开发站点，甚至</div>
</li>
<li><a href="/article/1950116322616602624.htm"
title="macOS 安装全攻略：从基础到企业级部署" target="_blank">macOS 安装全攻略：从基础到企业级部署</a>
<span class="text-muted">zqmgx13291</span>
<a class="tag" taget="_blank" href="/search/macos/1.htm">macos</a>
<div>引言：macOS安装的技术全景与价值macOS作为苹果生态的核心组件，其安装场景涵盖个人用户的系统重装、开发者的多系统环境搭建、企业级设备的批量部署等。据Apple官方数据，2024年全球活跃Mac设备超2亿台，其中AppleSilicon芯片机型占比达68%，架构差异（ARMvsx86）导致安装流程存在显著区别。本文将系统梳理macOS安装的全流程，从基础的单系统重装到复杂的双系统配置、硬件升级</div>
</li>
<li><a href="/article/1950089974804180992.htm"
title="9、Docker Compose 实战" target="_blank">9、Docker Compose 实战</a>
<span class="text-muted">小醉你真好</span>
<a class="tag" taget="_blank" href="/search/%23/1.htm">#</a><a class="tag" taget="_blank" href="/search/%E9%83%A8%E7%BD%B2%E4%B8%8D%E6%B1%82%E4%BA%BA/1.htm">部署不求人</a><a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/%E5%AE%B9%E5%99%A8/1.htm">容器</a><a class="tag" taget="_blank" href="/search/%E8%BF%90%E7%BB%B4/1.htm">运维</a>
<div>DockerCompose实战教程（含完整Nginx案例+配置项详解）适合读者：开发者、后端工程师、运维工程师、初学者环境要求：CentOS9+Docker已安装教程亮点：实战驱动、配置项详解、挂载说明、可直接复制使用标签：#Docker#DockerCompose#运维实战#Nginx部署一、什么是DockerCompose？DockerCompose是Docker官方推出的多容器应用编排工具，</div>
</li>
<li><a href="/article/1950087446070882304.htm"
title="《河北雄安新区规划纲要》" target="_blank">《河北雄安新区规划纲要》</a>
<span class="text-muted">郭强GQ</span>
<div>世界眼光、国际标准、中国特色、高点定位，生态优先、绿色发展，以人民为中心、保障和改善民生，保护弘扬中华优秀传统文化、延续历史文脉，符合党中央、国务院对雄安新区的战略定位和发展要求。科学构建城市空间布局雄安新区实行组团式发展，选择容城、安新两县交界区域作为起步区，先行开发建设启动区，稳步有序推进中期发展区建设，划定远期控制区为未来发展预留空间。城乡统筹、均衡发展、宜居宜业，形成“一主、五辅、多节点”</div>
</li>
<li><a href="/article/1950080525918072832.htm"
title="MySQL 数据类型详解" target="_blank">MySQL 数据类型详解</a>
<span class="text-muted">yimeixiaolangzai</span>
<a class="tag" taget="_blank" href="/search/MySQL/1.htm">MySQL</a><a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%BA%93/1.htm">数据库</a>
<div>在数据库设计和开发中，选择合适的数据类型对于存储和操作数据至关重要。MySQL提供了丰富的数据类型来满足不同的数据存储需求，这些数据类型可以分为数值类型、字符串类型、日期和时间类型，以及二进制类型。本文将详细介绍MySQL中的各类数据类型及其应用场景，帮助你更好地进行数据库设计。1.数值类型数值类型用于存储整数和浮点数，在处理数值运算时，这些数据类型扮演着关键角色。MySQL提供了多种数值类型，以</div>
</li>
<li><a href="/article/52.htm"
title="解读Servlet原理篇二---GenericServlet与HttpServlet" target="_blank">解读Servlet原理篇二---GenericServlet与HttpServlet</a>
<span class="text-muted">周凡杨</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/HttpServlet/1.htm">HttpServlet</a><a class="tag" taget="_blank" href="/search/%E6%BA%90%E7%90%86/1.htm">源理</a><a class="tag" taget="_blank" href="/search/GenericService/1.htm">GenericService</a><a class="tag" taget="_blank" href="/search/%E6%BA%90%E7%A0%81/1.htm">源码</a>
<div>在上一篇《解读Servlet原理篇一》中提到，要实现javax.servlet.Servlet接口（即写自己的Servlet应用），你可以写一个继承自javax.servlet.GenericServletr的generic Servlet ，也可以写一个继承自java.servlet.http.HttpServlet的HTTP Servlet（这就是为什么我们自定义的Servlet通常是exte</div>
</li>
<li><a href="/article/179.htm"
title="MySQL性能优化" target="_blank">MySQL性能优化</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%BA%93/1.htm">数据库</a><a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a>
<div> 性能优化是通过某些有效的方法来提高MySQL的运行速度，减少占用的磁盘空间。性能优化包含很多方面，例如优化查询速度，优化更新速度和优化MySQL服务器等。本文介绍方法的主要有：
a.优化查询
b.优化数据库结构
</div>
</li>
<li><a href="/article/306.htm"
title="ThreadPool定时重试" target="_blank">ThreadPool定时重试</a>
<span class="text-muted">dai_lm</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/ThreadPool/1.htm">ThreadPool</a><a class="tag" taget="_blank" href="/search/thread/1.htm">thread</a><a class="tag" taget="_blank" href="/search/timer/1.htm">timer</a><a class="tag" taget="_blank" href="/search/timertask/1.htm">timertask</a>
<div>项目需要当某事件触发时，执行http请求任务，失败时需要有重试机制，并根据失败次数的增加，重试间隔也相应增加，任务可能并发。
由于是耗时任务，首先考虑的就是用线程来实现，并且为了节约资源，因而选择线程池。
为了解决不定间隔的重试，选择Timer和TimerTask来完成
package threadpool;
public class ThreadPoolTest {
</div>
</li>
<li><a href="/article/433.htm"
title="Oracle 查看数据库的连接情况" target="_blank">Oracle 查看数据库的连接情况</a>
<span class="text-muted">周凡杨</span>
<a class="tag" taget="_blank" href="/search/sql/1.htm">sql</a><a class="tag" taget="_blank" href="/search/oracle+%E8%BF%9E%E6%8E%A5/1.htm">oracle 连接</a>
<div>首先要说的是，不同版本数据库提供的系统表会有不同，你可以根据数据字典查看该版本数据库所提供的表。
select * from dict where table_name like '%SESSION%';
就可以查出一些表，然后根据这些表就可以获得会话信息
select sid,serial#,status,username,schemaname,osuser,terminal,ma</div>
</li>
<li><a href="/article/560.htm"
title="类的继承" target="_blank">类的继承</a>
<span class="text-muted">朱辉辉33</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>类的继承可以提高代码的重用行，减少冗余代码；还能提高代码的扩展性。Java继承的关键字是extends
格式:public class 类名（子类）extends 类名（父类）{ }
子类可以继承到父类所有的属性和普通方法，但不能继承构造方法。且子类可以直接使用父类的public和
protected属性，但要使用private属性仍需通过调用。
子类的方法可以重写，但必须和父类的返回值类</div>
</li>
<li><a href="/article/687.htm"
title="android 悬浮窗特效" target="_blank">android 悬浮窗特效</a>
<span class="text-muted">肆无忌惮_</span>
<a class="tag" taget="_blank" href="/search/android/1.htm">android</a>
<div>最近在开发项目的时候需要做一个悬浮层的动画，类似于支付宝掉钱动画。但是区别在于，需求是浮出一个窗口，之后边缩放边位移至屏幕右下角标签处。效果图如下：
一开始考虑用自定义View来做。后来发现开线程让其移动很卡，ListView+动画也没法精确定位到目标点。
后来想利用Dialog的dismiss动画来完成。
自定义一个Dialog后，在styl</div>
</li>
<li><a href="/article/814.htm"
title="hadoop伪分布式搭建" target="_blank">hadoop伪分布式搭建</a>
<span class="text-muted">林鹤霄</span>
<a class="tag" taget="_blank" href="/search/hadoop/1.htm">hadoop</a>
<div>要修改4个文件 1: vim hadoop-env.sh 第九行 2: vim core-site.xml <configuration> &n</div>
</li>
<li><a href="/article/941.htm"
title="gdb调试命令" target="_blank">gdb调试命令</a>
<span class="text-muted">aigo</span>
<a class="tag" taget="_blank" href="/search/gdb/1.htm">gdb</a>
<div>原文：http://blog.csdn.net/hanchaoman/article/details/5517362
一、GDB常用命令简介
r run 运行.程序还没有运行前使用 c cuntinue </div>
</li>
<li><a href="/article/1068.htm"
title="Socket编程的HelloWorld实例" target="_blank">Socket编程的HelloWorld实例</a>
<span class="text-muted">alleni123</span>
<a class="tag" taget="_blank" href="/search/socket/1.htm">socket</a>
<div>public class Client
{
public static void main(String[] args)
{
Client c=new Client();
c.receiveMessage();
}
public void receiveMessage(){
Socket s=null;
BufferedRea</div>
</li>
<li><a href="/article/1195.htm"
title="线程同步和异步" target="_blank">线程同步和异步</a>
<span class="text-muted">百合不是茶</span>
<a class="tag" taget="_blank" href="/search/%E7%BA%BF%E7%A8%8B%E5%90%8C%E6%AD%A5/1.htm">线程同步</a><a class="tag" taget="_blank" href="/search/%E5%BC%82%E6%AD%A5/1.htm">异步</a>
<div>多线程和同步 : 如进程、线程同步，可理解为进程或线程A和B一块配合，A执行到一定程度时要依靠B的某个结果，于是停下来，示意B运行；B依言执行，再将结果给A；A再继续操作。 所谓同步，就是在发出一个功能调用时，在没有得到结果之前，该调用就不返回，同时其它线程也不能调用这个方法
多线程和异步:多线程可以做不同的事情,涉及到线程通知
&</div>
</li>
<li><a href="/article/1322.htm"
title="JSP中文乱码分析" target="_blank">JSP中文乱码分析</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/jsp/1.htm">jsp</a><a class="tag" taget="_blank" href="/search/%E4%B8%AD%E6%96%87%E4%B9%B1%E7%A0%81/1.htm">中文乱码</a>
<div> 在JSP的开发过程中，经常出现中文乱码的问题。
首先了解一下Java中文问题的由来：
Java的内核和class文件是基于unicode的，这使Java程序具有良好的跨平台性，但也带来了一些中文乱码问题的麻烦。原因主要有两方面，</div>
</li>
<li><a href="/article/1449.htm"
title="js实现页面跳转重定向的几种方式" target="_blank">js实现页面跳转重定向的几种方式</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a><a class="tag" taget="_blank" href="/search/%E9%87%8D%E5%AE%9A%E5%90%91/1.htm">重定向</a>
<div> js实现页面跳转重定向有如下几种方式：
一.window.location.href
<script language="javascript"type="text/javascript">
window.location.href="http://www.baidu.c</div>
</li>
<li><a href="/article/1576.htm"
title="【Struts2三】Struts2 Action转发类型" target="_blank">【Struts2三】Struts2 Action转发类型</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/struts2/1.htm">struts2</a>
<div> 在【Struts2一】 Struts Hello World http://bit1129.iteye.com/blog/2109365中配置了一个简单的Action，配置如下
<!DOCTYPE struts PUBLIC
"-//Apache Software Foundation//DTD Struts Configurat</div>
</li>
<li><a href="/article/1703.htm"
title="【HBase十一】Java API操作HBase" target="_blank">【HBase十一】Java API操作HBase</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/hbase/1.htm">hbase</a>
<div>Admin类的主要方法注释：
1. 创建表
/**
* Creates a new table. Synchronous operation.
*
* @param desc table descriptor for table
* @throws IllegalArgumentException if the table name is res</div>
</li>
<li><a href="/article/1830.htm"
title="nginx gzip" target="_blank">nginx gzip</a>
<span class="text-muted">ronin47</span>
<a class="tag" taget="_blank" href="/search/nginx+gzip/1.htm">nginx gzip</a>
<div>Nginx GZip 压缩
Nginx GZip 模块文档详见：http://wiki.nginx.org/HttpGzipModule
常用配置片段如下：
gzip on; gzip_comp_level 2; # 压缩比例，比例越大，压缩时间越长。默认是1 gzip_types text/css text/javascript; # 哪些文件可以被压缩 gzip_disable &q</div>
</li>
<li><a href="/article/1957.htm"
title="java-7.微软亚院之编程判断俩个链表是否相交 给出俩个单向链表的头指针，比如 h1 ， h2 ，判断这俩个链表是否相交" target="_blank">java-7.微软亚院之编程判断俩个链表是否相交 给出俩个单向链表的头指针，比如 h1 ， h2 ，判断这俩个链表是否相交</a>
<span class="text-muted">bylijinnan</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>
public class LinkListTest {
/**
* we deal with two main missions:
*
* A.
* 1.we create two joined-List(both have no loop)
* 2.whether list1 and list2 join
* 3.print the join</div>
</li>
<li><a href="/article/2084.htm"
title="Spring源码学习-JdbcTemplate batchUpdate批量操作" target="_blank">Spring源码学习-JdbcTemplate batchUpdate批量操作</a>
<span class="text-muted">bylijinnan</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/spring/1.htm">spring</a>
<div>Spring JdbcTemplate的batch操作最后还是利用了JDBC提供的方法，Spring只是做了一下改造和封装
JDBC的batch操作：
String sql = "INSERT INTO CUSTOMER " +
"(CUST_ID, NAME, AGE) VALUES (?, ?, ?)";
</div>
</li>
<li><a href="/article/2211.htm"
title="[JWFD开源工作流]大规模拓扑矩阵存储结构最新进展" target="_blank">[JWFD开源工作流]大规模拓扑矩阵存储结构最新进展</a>
<span class="text-muted">comsci</span>
<a class="tag" taget="_blank" href="/search/%E5%B7%A5%E4%BD%9C%E6%B5%81/1.htm">工作流</a>
<div> 生成和创建类已经完成,构造一个100万个元素的矩阵模型,存储空间只有11M大,请大家参考我在博客园上面的文档"构造下一代工作流存储结构的尝试",更加相信的设计和代码将陆续推出.........
竞争对手的能力也很强.......,我相信..你们一定能够先于我们推出大规模拓扑扫描和分析系统的....</div>
</li>
<li><a href="/article/2338.htm"
title="base64编码和url编码" target="_blank">base64编码和url编码</a>
<span class="text-muted">cuityang</span>
<a class="tag" taget="_blank" href="/search/base64/1.htm">base64</a><a class="tag" taget="_blank" href="/search/url/1.htm">url</a>
<div>import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.io.UnsupportedEncodingException; </div>
</li>
<li><a href="/article/2465.htm"
title="web应用集群Session保持" target="_blank">web应用集群Session保持</a>
<span class="text-muted">dalan_123</span>
<a class="tag" taget="_blank" href="/search/session/1.htm">session</a>
<div>关于使用 memcached 或redis 存储 session ，以及使用 terracotta 服务器共享。建议使用 redis，不仅仅因为它可以将缓存的内容持久化，还因为它支持的单个对象比较大，而且数据类型丰富，不只是缓存 session，还可以做其他用途，一举几得啊。1、使用 filter 方法存储这种方法比较推荐，因为它的服务器使用范围比较多，不仅限于tomcat ，而且实现的原理比较简</div>
</li>
<li><a href="/article/2719.htm"
title="Yii 框架里数据库操作详解-[增加、查询、更新、删除的方法 'AR模式']" target="_blank">Yii 框架里数据库操作详解-[增加、查询、更新、删除的方法 'AR模式']</a>
<span class="text-muted">dcj3sjt126com</span>
<a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%BA%93/1.htm">数据库</a>
<div> public function getMinLimit () { $sql = "..."; $result = yii::app()->db->createCo</div>
</li>
<li><a href="/article/2846.htm"
title="solr StatsComponent（聚合统计）" target="_blank">solr StatsComponent（聚合统计）</a>
<span class="text-muted">eksliang</span>
<a class="tag" taget="_blank" href="/search/solr%E8%81%9A%E5%90%88%E6%9F%A5%E8%AF%A2/1.htm">solr聚合查询</a><a class="tag" taget="_blank" href="/search/solr+stats/1.htm">solr stats</a>
<div>StatsComponent
转载请出自出处：http://eksliang.iteye.com/blog/2169134
http://eksliang.iteye.com/ 一、概述
Solr可以利用StatsComponent 实现数据库的聚合统计查询，也就是min、max、avg、count、sum的功能
二、参数</div>
</li>
<li><a href="/article/2973.htm"
title="百度一道面试题" target="_blank">百度一道面试题</a>
<span class="text-muted">greemranqq</span>
<a class="tag" taget="_blank" href="/search/%E4%BD%8D%E8%BF%90%E7%AE%97/1.htm">位运算</a><a class="tag" taget="_blank" href="/search/%E7%99%BE%E5%BA%A6%E9%9D%A2%E8%AF%95/1.htm">百度面试</a><a class="tag" taget="_blank" href="/search/%E5%AF%BB%E6%89%BE%E5%A5%87%E6%95%B0%E7%AE%97%E6%B3%95/1.htm">寻找奇数算法</a><a class="tag" taget="_blank" href="/search/bitmap+%E7%AE%97%E6%B3%95/1.htm">bitmap 算法</a>
<div>那天看朋友提了一个百度面试的题目：怎么找出{1,1,2,3,3,4,4,4,5,5,5,5} 找出出现次数为奇数的数字.
我这里复制的是原话，当然顺序是不一定的，很多拿到题目第一反应就是用map,当然可以解决，但是效率不高。
还有人觉得应该用算法xxx,我是没想到用啥算法好...！
还有觉得应该先排序...
还有觉</div>
</li>
<li><a href="/article/3100.htm"
title="Spring之在开发中使用SpringJDBC" target="_blank">Spring之在开发中使用SpringJDBC</a>
<span class="text-muted">ihuning</span>
<a class="tag" taget="_blank" href="/search/spring/1.htm">spring</a>
<div>
在实际开发中使用SpringJDBC有两种方式：
1. 在Dao中添加属性JdbcTemplate并用Spring注入；
JdbcTemplate类被设计成为线程安全的，所以可以在IOC 容器中声明它的单个实例，并将这个实例注入到所有的 DAO 实例中。JdbcTemplate也利用了Java 1.5 的特定(自动装箱，泛型，可变长度</div>
</li>
<li><a href="/article/3227.htm"
title="JSON API 1.0 核心开发者自述 | 你所不知道的那些技术细节" target="_blank">JSON API 1.0 核心开发者自述 | 你所不知道的那些技术细节</a>
<span class="text-muted">justjavac</span>
<a class="tag" taget="_blank" href="/search/json/1.htm">json</a>
<div>2013年5月，Yehuda Katz 完成了JSON API(英文，中文) 技术规范的初稿。事情就发生在 RailsConf 之后，在那次会议上他和 Steve Klabnik 就 JSON 雏形的技术细节相聊甚欢。在沟通单一 Rails 服务器库—— ActiveModel::Serializers 和单一 JavaScript 客户端库——&</div>
</li>
<li><a href="/article/3354.htm"
title="网站项目建设流程概述" target="_blank">网站项目建设流程概述</a>
<span class="text-muted">macroli</span>
<a class="tag" taget="_blank" href="/search/%E5%B7%A5%E4%BD%9C/1.htm">工作</a>
<div>一.概念
网站项目管理就是根据特定的规范、在预算范围内、按时完成的网站开发任务。
二.需求分析
项目立项
我们接到客户的业务咨询，经过双方不断的接洽和了解，并通过基本的可行性讨论够，初步达成制作协议，这时就需要将项目立项。较好的做法是成立一个专门的项目小组，小组成员包括：项目经理，网页设计，程序员，测试员，编辑/文档等必须人员。项目实行项目经理制。
客户的需求说明书
第一步是需</div>
</li>
<li><a href="/article/3481.htm"
title="AngularJs 三目运算 表达式判断" target="_blank">AngularJs 三目运算 表达式判断</a>
<span class="text-muted">qiaolevip</span>
<a class="tag" taget="_blank" href="/search/%E6%AF%8F%E5%A4%A9%E8%BF%9B%E6%AD%A5%E4%B8%80%E7%82%B9%E7%82%B9/1.htm">每天进步一点点</a><a class="tag" taget="_blank" href="/search/%E5%AD%A6%E4%B9%A0%E6%B0%B8%E6%97%A0%E6%AD%A2%E5%A2%83/1.htm">学习永无止境</a><a class="tag" taget="_blank" href="/search/%E4%BC%97%E8%A7%82%E5%8D%83%E8%B1%A1/1.htm">众观千象</a><a class="tag" taget="_blank" href="/search/AngularJS/1.htm">AngularJS</a>
<div>事件回顾：由于需要修改同一个模板，里面包含2个不同的内容，第一个里面使用的时间差和第二个里面名称不一样，其他过滤器，内容都大同小异。希望杜绝If这样比较傻的来判断if-show or not，继续追究其源码。
var b = "{{",
a = "}}";
this.startSymbol = function(a) {
</div>
</li>
<li><a href="/article/3608.htm"
title="Spark算子：统计RDD分区中的元素及数量" target="_blank">Spark算子：统计RDD分区中的元素及数量</a>
<span class="text-muted">superlxw1234</span>
<a class="tag" taget="_blank" href="/search/spark/1.htm">spark</a><a class="tag" taget="_blank" href="/search/spark%E7%AE%97%E5%AD%90/1.htm">spark算子</a><a class="tag" taget="_blank" href="/search/Spark+RDD%E5%88%86%E5%8C%BA%E5%85%83%E7%B4%A0/1.htm">Spark RDD分区元素</a>
<div>关键字：Spark算子、Spark RDD分区、Spark RDD分区元素数量
Spark RDD是被分区的，在生成RDD时候，一般可以指定分区的数量，如果不指定分区数量，当RDD从集合创建时候，则默认为该程序所分配到的资源的CPU核数，如果是从HDFS文件创建，默认为文件的Block数。
可以利用RDD的mapPartitionsWithInd</div>
</li>
<li><a href="/article/3735.htm"
title="Spring 3.2.x将于2016年12月31日停止支持" target="_blank">Spring 3.2.x将于2016年12月31日停止支持</a>
<span class="text-muted">wiselyman</span>
<a class="tag" taget="_blank" href="/search/Spring+3/1.htm">Spring 3</a>
<div>
Spring 团队公布在2016年12月31日停止对Spring Framework 3.2.x（包含tomcat 6.x）的支持。在此之前spring团队将持续发布3.2.x的维护版本。
请大家及时准备及时升级到Spring </div>
</li>
<li><a href="/article/3862.htm"
title="fis纯前端解决方案fis-pure" target="_blank">fis纯前端解决方案fis-pure</a>
<span class="text-muted">zccst</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a>
<div>作者：zccst
FIS通过插件扩展可以完美的支持模块化的前端开发方案，我们通过FIS的二次封装能力，封装了一个功能完备的纯前端模块化方案pure。
1，fis-pure的安装
$ fis install -g fis-pure
$ pure -v
0.1.4
2，下载demo到本地
git clone https://github.com/hefangshi/f</div>
</li>
</ul>
</div>
</div>
</div>
<div>
<div class="container">
<div class="indexes">
<strong>按字母分类：</strong>
<a href="/tags/A/1.htm" target="_blank">A</a><a href="/tags/B/1.htm" target="_blank">B</a><a href="/tags/C/1.htm" target="_blank">C</a><a
href="/tags/D/1.htm" target="_blank">D</a><a href="/tags/E/1.htm" target="_blank">E</a><a href="/tags/F/1.htm" target="_blank">F</a><a
href="/tags/G/1.htm" target="_blank">G</a><a href="/tags/H/1.htm" target="_blank">H</a><a href="/tags/I/1.htm" target="_blank">I</a><a
href="/tags/J/1.htm" target="_blank">J</a><a href="/tags/K/1.htm" target="_blank">K</a><a href="/tags/L/1.htm" target="_blank">L</a><a
href="/tags/M/1.htm" target="_blank">M</a><a href="/tags/N/1.htm" target="_blank">N</a><a href="/tags/O/1.htm" target="_blank">O</a><a
href="/tags/P/1.htm" target="_blank">P</a><a href="/tags/Q/1.htm" target="_blank">Q</a><a href="/tags/R/1.htm" target="_blank">R</a><a
href="/tags/S/1.htm" target="_blank">S</a><a href="/tags/T/1.htm" target="_blank">T</a><a href="/tags/U/1.htm" target="_blank">U</a><a
href="/tags/V/1.htm" target="_blank">V</a><a href="/tags/W/1.htm" target="_blank">W</a><a href="/tags/X/1.htm" target="_blank">X</a><a
href="/tags/Y/1.htm" target="_blank">Y</a><a href="/tags/Z/1.htm" target="_blank">Z</a><a href="/tags/0/1.htm" target="_blank">其他</a>
</div>
</div>
</div>
<footer id="footer" class="mb30 mt30">
<div class="container">
<div class="footBglm">
<a target="_blank" href="/">首页</a> -
<a target="_blank" href="/custom/about.htm">关于我们</a> -
<a target="_blank" href="/search/Java/1.htm">站内搜索</a> -
<a target="_blank" href="/sitemap.txt">Sitemap</a> -
<a target="_blank" href="/custom/delete.htm">侵权投诉</a>
</div>
<div class="copyright">版权所有 IT知识库 CopyRight © 2000-2050 E-COM-NET.COM , All Rights Reserved.
<!-- <a href="https://beian.miit.gov.cn/" rel="nofollow" target="_blank">京ICP备09083238号</a><br>-->
</div>
</div>
</footer>
<!-- 代码高亮 -->
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shCore.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shLegacy.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shAutoloader.js"></script>
<link type="text/css" rel="stylesheet" href="/static/syntaxhighlighter/styles/shCoreDefault.css"/>
<script type="text/javascript" src="/static/syntaxhighlighter/src/my_start_1.js"></script>
</body>
</html><script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script>