k8s-1.24.4详细安装教程（附镜像）
- 废话不多说，直接介绍怎么安装（这里是安装的CICD环境，后续会一步一步实现gitlab+docker+harbor+k8s+jenkins，不需要的软件，可以直接忽略。）
环境配置
| Hostname | IP | 备注 （为节省虚拟机，安装的其他软件，只装k8s可不安装） |
|---|---|---|
| master | 192.168.47.100 | jenkins(至少3G内存) |
| node1 | 192.168.47.101 | harbor |
| node2 | 192.168.47.102 | gitlab（至少4G内存） |
- 需要注意版本之间的依赖关系
| 软件 | 版本 | 备注 |
|---|---|---|
| 操作系统 | CentOS7 | / |
| 内核 | 5.19.8 | 如果内核为3.10，建议先升级（master初始化可能会报错） |
| JDK | 1.8，11 | https://www.oracle.com/java/technologies/javase-downloads.html |
| Maven | 3.0.5 | https://maven.apache.org/ |
| docker | 20.10.17 | https://www.docker.com/ |
| Kubernetes | 1.24.4 | https://kubernetes.io/ |
| cri-docker | 0.2.3 | |
| Gitlab | 15.4 | https://about.gitlab.com/ |
| Jenkins | 2.361.1-1.1 | 依赖jdk11 |
| Harbor | v2.6.0 | https://goharbor.io/ |
- 查看以及升级内核的方法
查看内核版本
[root@k8s cgroup]# uname -r
3.10.0-327.el7.x86_64
导入EL repo库
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
[root@k8s cgroup]# yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
可安装的软件包
elrepo-release.noarch
kernel-lt.x86_64
kernel-lt-devel.x86_64
kernel-lt-doc.noarch
kernel-lt-headers.x86_64
kernel-lt-tools.x86_64
kernel-lt-tools-libs.x86_64
kernel-lt-tools-libs-devel.x86_64
kernel-ml.x86_64
kernel-ml-devel.x86_64
kernel-ml-doc.noarch
kernel-ml-headers.x86_64
kernel-ml-tools.x86_64
kernel-ml-tools-libs.x86_64
kernel-ml-tools-libs-devel.x86_64
perf.x86_64
python-perf.x86_64
安装稳定内核
yum --enablerepo=elrepo-kernel install kernel-ml
修改GRUB选项，使新安装的内核作为优先启动
cp /etc/default/grub /etc/default/grub_bak
[root@k8s cgroup]# vim /etc/default/grub
[root@k8s cgroup]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
# GRUB_DEFAULT=saved-->GRUB_DEFAULT=0
GRUB_DEFAULT=0
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
重新创建内核配置
[root@k8s cgroup]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.19.8-1.el7.elrepo.x86_64
Found initrd image: /boot/initramfs-5.19.8-1.el7.elrepo.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-327.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-327.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-575e24cf52124364b060acab9088f9b8
Found initrd image: /boot/initramfs-0-rescue-575e24cf52124364b060acab9088f9b8.img
done
重启验证
reboot
uname -r
5.19.8-1.el7.elrepo.x86_64
- 环境配置
#!/bin/bash
# 彻底清除原先安装的k8s环境
kubeadm reset -f
modprobe -r ipip
rm -rf ~/.kube/
rm -rf /etc/kubernetes/
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
rm -rf /usr/bin/kube*
rm -rf /etc/cni
rm -rf /opt/cni
rm -rf /var/lib/etcd
rm -rf /var/etcd
yum clean all
yum remove kube* -y
# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
firewall-cmd --state
# 关闭selinux
setenforce 0
sed -i "s/SELINUX=.*/SELINUX=disabled/g" /etc/selinux/config
# 关闭swap
# 临时关闭
swapoff -a
# 永久关闭,这个需要重启生效
sed -i 's/\/dev\/mapper\/centos-swap/#\/dev\/mapper\/centos-swap/g' /etc/fstab
# 允许桥接
cat > /etc/modules-load.d/k8s.conf <cat > /etc/sysctl.d/k8s.conf <.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
# hosts文件 域名通信
echo 192.168.47.100 k8s-master >> /etc/hosts
echo 192.168.47.101 k8s-node >> /etc/hosts
echo 192.168.47.102 k8s-node2 >> /etc/hosts
# 配置阿里源
cd /etc/yum.repos.d/
mkdir bak
mv * bak
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all
yum makecache
# 安装docker
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
mkdir /etc/docker
cat > /etc/docker/daemon.json <{
"registry-mirrors": ["https://hzfyo6gg.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
yum install -y yum-utils device-mapper-persistent-data lvm2
yum install -y docker-ce
systemctl enable docker
systemctl daemon-reload
systemctl restart docker
# 安装k8s-1.24.4
cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF
yum install kubeadm-1.24.4 kubelet-1.24.4 -y
systemctl enable kubelet
- 安装jdk和maven（用于cicd流程，可选）
yum install -y java-openjdk
yum install -y maven
mkdir /home/repository
vim /etc/maven/settings.xml
"http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
# 加上
/home/repository</localRepository>
# 修改
aliyunmaven</id>
*</mirrorOf>
阿里云公共仓库</name
https://maven.aliyun.com/repository/public</url>
</mirror>
</mirrors>
k8s集群搭建
安装和配置cri-docker
- 软件包–>百度网盘
- 链接：https://pan.baidu.com/s/1DztUfYpEZdkzic7lg1yF1g
提取码：raq6
tar -xf cri-dockerd-0.2.3.amd64.tgz
cp cri-dockerd/cri-dockerd /usr/local/
-
创建cri-docker启动文件
https://github.com/Mirantis/cri-dockerd/tree/master/packaging/systemd/cri-docker.servicehttps://github.com/Mirantis/cri-dockerd/blob/master/packaging/systemd/cri-docker.socket
[root@k8s ~]# vim /usr/lib/systemd/system/cri-docker.service
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
# 指定pause:3.7
ExecStart=/usr/local/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
[root@k8s ~]# vim /usr/lib/systemd/system/cri-docker.socket
[Unit]
Description=CRI Docker Socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target
- 启动cri-docker并设置开机自动启动
systemctl daemon-reload
systemctl restart cri-docker.service
systemctl enable cri-docker.service
初始化master
- 必须指定–cri-socket unix://var/run/cri-dockerd.sock()，除了kubeadm init命令外，kubeadm reset，kubeadm join都要指定
kubeadm init --image-repository registry.aliyuncs.com/google_containers --cri-socket unix://var/run/cri-dockerd.sock --ignore-preflight-errors=NumCPU --service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16 --kubernetes-version=1.24.4
# 这里是因为内核版本过低报错，如果升级过内核，请忽略
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR SystemVerification]: unexpected kernel config: CONFIG_CGROUP_PIDS
[ERROR SystemVerification]: missing required cgroups: pids
报错：[ERROR SystemVerification]: unexpected kernel config: CONFIG_CGROUP_PIDS
[ERROR SystemVerification]: missing required cgroups: pids解决：升级内核
[root@k8s cgroup]# uname -r 3.10.0-327.el7.x86_64 导入EL repo库 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm [root@k8s cgroup]# yum --disablerepo="*" --enablerepo="elrepo-kernel" list available 可安装的软件包 elrepo-release.noarch kernel-lt.x86_64 kernel-lt-devel.x86_64 kernel-lt-doc.noarch kernel-lt-headers.x86_64 kernel-lt-tools.x86_64 kernel-lt-tools-libs.x86_64 kernel-lt-tools-libs-devel.x86_64 kernel-ml.x86_64 kernel-ml-devel.x86_64 kernel-ml-doc.noarch kernel-ml-headers.x86_64 kernel-ml-tools.x86_64 kernel-ml-tools-libs.x86_64 kernel-ml-tools-libs-devel.x86_64 perf.x86_64 python-perf.x86_64 安装稳定内核 yum --enablerepo=elrepo-kernel install kernel-ml 修改GRUB选项，使新安装的内核作为优先启动 cp /etc/default/grub /etc/default/grub_bak [root@k8s cgroup]# vim /etc/default/grub [root@k8s cgroup]# cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" # GRUB_DEFAULT=saved-->GRUB_DEFAULT=0 GRUB_DEFAULT=0 GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet" GRUB_DISABLE_RECOVERY="true" 重新创建内核配置 [root@k8s cgroup]# grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... Found linux image: /boot/vmlinuz-5.19.8-1.el7.elrepo.x86_64 Found initrd image: /boot/initramfs-5.19.8-1.el7.elrepo.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-327.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-327.el7.x86_64.img Found linux image: /boot/vmlinuz-0-rescue-575e24cf52124364b060acab9088f9b8 Found initrd image: /boot/initramfs-0-rescue-575e24cf52124364b060acab9088f9b8.img done 重启验证 reboot uname -r 5.19.8-1.el7.elrepo.x86_64
kubeadm init --image-repository registry.aliyuncs.com/google_containers --cri-socket unix://var/run/cri-dockerd.sock --ignore-preflight-errors=NumCPU --service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16 --kubernetes-version=1.24.4
# 这里是因为镜像太大，没能拉取成功。可以直接导入我网盘里的镜像或者手动拉取需要的镜像
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image registry.aliyuncs.com/google_containers/etcd:3.5.3-0: output: E0914 13:11:22.857372 17990 remote_image.go:218] "PullImage from image service failed" err="rpc error: code = Unknown desc = context deadline exceeded" image="registry.aliyuncs.com/google_containers/etcd:3.5.3-0"
time="2022-09-14T13:11:22+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = context deadline exceeded"
, error: exit status 1
报错：[ERROR ImagePull]: failed to pull image registry.aliyuncs.com/google_containers/etcd:3.5.3-0: output: E0914 13:11:22.857372 17990 remote_image.go:218] “PullImage from image service failed” err=“rpc error: code = Unknown desc = context deadline exceeded” image=“registry.aliyuncs.com/google_containers/etcd:3.5.3-0”
原因：镜像etcd:3.5.3-0无法下载
解决：手动下载
查看需要的镜像 [root@k8s ~]# kubeadm config images list I0914 14:02:13.369582 21306 version.go:255] remote version is much newer: v1.25.0; falling back to: stable-1.24 k8s.gcr.io/kube-apiserver:v1.24.4 k8s.gcr.io/kube-controller-manager:v1.24.4 k8s.gcr.io/kube-scheduler:v1.24.4 k8s.gcr.io/kube-proxy:v1.24.4 k8s.gcr.io/pause:3.7 k8s.gcr.io/etcd:3.5.3-0 k8s.gcr.io/coredns/coredns:v1.8.6 拉取需要的镜像 [root@k8s ~]# docker pull registry.aliyuncs.com/google_containers/etcd:3.5.3-0 3.5.3-0: Pulling from google_containers/etcd 36698cfa5275: Pull complete 924f6cbb1ab3: Pull complete 11ade7be2717: Pull complete 8c6339f7974a: Pull complete d846fbeccd2d: Pull complete Digest: sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 Status: Downloaded newer image for registry.aliyuncs.com/google_containers/etcd:3.5.3-0 registry.aliyuncs.com/google_containers/etcd:3.5.3-0
[root@k8s ~]# kubeadm init --image-repository registry.aliyuncs.com/google_containers --cri-socket unix://var/run/cri-dockerd.sock --ignore-preflight-errors=NumCPU --service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16 、--kubernetes-version=1.24.4
[kubelet-check] Initial timeout of 40s passed.
error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: nodes "k8s" not found
To see the stack trace of this error execute with --v=5 or higher
报错：error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: nodes “k8s” not found
解决:
swapoff -a && kubeadm reset 命令：kubeadm reset 报错：Found multiple CRI endpoints on the host. Please define which one do you wish to use by setting the 'criSocket' field in the kubeadm configuration file: unix:///var/run/containerd/containerd.sock, unix:///var/run/cri-dockerd.sock 解决：kubeadm reset --cri-socket unix://var/run/cri-dockerd.sock swapoff -a && kubeadm reset --cri-socket unix://var/run/cri-dockerd.sock systemctl daemon-reload && systemctl restart kubelet && iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
[root@k8s ~]# kubeadm init --image-repository registry.aliyuncs.com/google_containers --cri-socket unix://var/run/cri-dockerd.sock --ignore-preflight-errors=NumCPU --service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16 --kubernetes-version=1.24.4
I0914 14:56:03.338660 39811 version.go:255] remote version is much newer: v1.25.0; falling back to: stable-1.24
[init] Using Kubernetes version: v1.24.4
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.10.0.1 192.168.47.50]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s localhost] and IPs [192.168.47.50 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s localhost] and IPs [192.168.47.50 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 8.503409 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: j5h4wu.crmpe07onlh5h0o8
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.47.50:6443 --token j5h4wu.crmpe07onlh5h0o8 \
--discovery-token-ca-cert-hash sha256:0af083946d013ec301d311bc19a784be123f5baa15ca7cb2de12c292e288635e
按要求执行
[root@k8s ~]# mkdir -p $HOME/.kube
[root@k8s ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s ~]# chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s ~]# export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl命令补全
yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
安装网络flannel
- 安装 flannel网络插件
echo 52.74.223.119 github.com >> /etc/hosts
echo 192.30.253.119 gist.github.com >> /etc/hosts
echo 54.169.195.247 api.github.com >> /etc/hosts
echo 185.199.111.153 assets-cdn.github.com >> /etc/hosts
echo 151.101.64.133 raw.githubusercontent.com >> /etc/hosts
echo 151.101.108.133 user-images.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 gist.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 cloud.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 camo.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars0.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars1.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars2.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars3.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars4.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars5.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars6.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars7.githubusercontent.com >> /etc/hosts
echo 151.101.76.133 avatars8.githubusercontent.com >> /etc/hosts
[root@k8s ~]# wget --no-check-certificate https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
[root@k8s ~]# vim kube-flannel.yml
net-conf.json: |
{
"Network": "10.122.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
[root@k8s ~]# kubectl apply -f kube-flannel.yml
[root@k8s ~]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-6t62g 1/1 Running 0 30s
kube-system coredns-74586cf9b6-jbvtt 0/1 ContainerCreating 0 10m
kube-system coredns-74586cf9b6-q5mlw 0/1 ContainerCreating 0 157m
kube-system etcd-k8s 1/1 Running 1 (38m ago) 158m
kube-system kube-apiserver-k8s 1/1 Running 1 (37m ago) 158m
kube-system kube-controller-manager-k8s 1/1 Running 8 (7m20s ago) 158m
kube-system kube-proxy-qdm7z 1/1 Running 1 (38m ago) 157m
kube-system kube-scheduler-k8s 1/1 Running 8 (7m20s ago) 158m
[root@k8s ~]# kubectl describe pod coredns-74586cf9b6-jbvtt -n kube-system
Warning FailedCreatePodSandBox 4m46s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "23afc3b1f16600c4139d25c54f0cc0f802ddc356b11d3962d405d2892d0b44bb" network for pod "coredns-74586cf9b6-jbvtt": networkPlugin cni failed to set up pod "coredns-74586cf9b6-jbvtt_kube-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized, failed to clean up sandbox container "23afc3b1f16600c4139d25c54f0cc0f802ddc356b11d3962d405d2892d0b44bb" network for pod "coredns-74586cf9b6-jbvtt": networkPlugin cni failed to teardown pod "coredns-74586cf9b6-jbvtt_kube-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized]
报错：Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container “23afc3b1f16600c4139d25c54f0cc0f802ddc356b11d3962d405d2892d0b44bb” network for pod “coredns-74586cf9b6-jbvtt”: networkPlugin cni failed to set up pod “coredns-74586cf9b6-jbvtt_kube-system” network: error getting ClusterInformation: connection is unauthorized: Unauthorized, failed to clean up sandbox container “23afc3b1f16600c4139d25c54f0cc0f802ddc356b11d3962d405d2892d0b44bb” network for pod “coredns-74586cf9b6-jbvtt”: networkPlugin cni failed to teardown pod “coredns-74586cf9b6-jbvtt_kube-system” network: error getting ClusterInformation: connection is unauthorized: Unauthorized]
原因：卸载calico有残留
解决：
[root@k8s ~]# ipvsadm --clear [root@k8s ~]# rm -rf /etc/cni/net.d/ [root@k8s ~]# kubectl delete -f kube-flannel.yml [root@k8s ~]# kubectl create -f kube-flannel.yml [root@k8s ~]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-flannel kube-flannel-ds-6t62g 1/1 Running 0 36s kube-system coredns-74586cf9b6-jbvtt 1/1 Running 0 10m kube-system coredns-74586cf9b6-q5mlw 1/1 Running 0 157m kube-system etcd-k8s 1/1 Running 1 (38m ago) 158m kube-system kube-apiserver-k8s 1/1 Running 1 (38m ago) 158m kube-system kube-controller-manager-k8s 1/1 Running 8 (7m26s ago) 158m kube-system kube-proxy-qdm7z 1/1 Running 1 (38m ago) 157m kube-system kube-scheduler-k8s 1/1 Running 8 (7m26s ago) 158m
子节点配置
查看加入集群的命令
[root@k8s ~]# kubeadm token create --print-join-command
kubeadm join 192.168.47.50:6443 --token 0i4x0o.zkrwba68j2pf19um --discovery-token-ca-cert-hash sha256:473f633d586094fc423a1c1544fbef44dc46c736c747b8b6bfc6343207e10650
加入集群
kubeadm join 192.168.47.50:6443 --token j5h4wu.crmpe07onlh5h0o8 --discovery-token-ca-cert-hash sha256:0af083946d013ec301d311bc19a784be123f5baa15ca7cb2de12c292e288635e --cri-socket unix://var/run/cri-dockerd.sock
[root@k8s ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane 170m v1.24.4
k8s-node NotReady 74s v1.24.4
k8s-node2 NotReady 54s v1.24.4
查看日志
journalctl -xeu kubelet -l
Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized"
报错：Container runtime network not ready" networkReady=“NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized”
解决：拷贝master上/etc/cni/net.d 目录下的文件到有问题的节点上
mkdir -p /etc/cni/net.d/ scp k8s:/etc/cni/net.d/* /etc/cni/net.d/
[root@k8s ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane 3h2m v1.24.4
k8s-node Ready 12m v1.24.4
k8s-node2 Ready 12m v1.24.4
安装ingress-nginx
- 下载yaml
mkdir -p nginx-ingress
cd nginx-ingress
curl -O https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.2.0/deploy/static/provider/cloud/deploy.yaml
- yaml内容（这里是解决报错后的版本，可以直接复制）
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- ""
resources:
- configmaps
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: bitnami/nginx-ingress-controller
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: dyrnq/kube-webhook-certgen:v1.1.1
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: dyrnq/kube-webhook-certgen:v1.1.1
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
提前拉镜像（如果需要，可以修改镜像名）
[root@k8s ~]# docker search nginx-ingress-controller
[root@k8s ~]# docker pull bitnami/nginx-ingress-controller
[root@k8s ~]# docker images
bitnami/nginx-ingress-controller latest 80ae81101d41 8 months ago 388MB
[root@k8s ~]# docker tag bitnami/nginx-ingress-controller:latest bitnami/nginx-ingress-controller:v1.24.4
[root@k8s ~]# vim ingress-nginx.yaml
containers:
- name: nginx-ingress-controller
image: bitnami/nginx-ingress-controller:v1.24.4
[root@k8s-node2 ~]# docker pull dyrnq/kube-webhook-certgen:v1.1.1
v1.1.1: Pulling from dyrnq/kube-webhook-certgen
ec52731e9273: Pull complete
b90aa28117d4: Pull complete
Digest: sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
Status: Downloaded newer image for dyrnq/kube-webhook-certgen:v1.1.1
docker.io/dyrnq/kube-webhook-certgen:v1.1.1
[root@k8s nginx-ingress]# kubectl apply -f deploy.yaml
namespace/ingress-nginx configured
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
[root@k8s nginx-ingress]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-zpscz 0/1 Completed 0 10m
ingress-nginx-admission-patch-fbcrj 0/1 Completed 0 10m
ingress-nginx-controller-599c66746c-g2rmw 1/1 Running 0 7m41s
查看日志
[root@k8s nginx-ingress]# kubectl logs -n ingress-nginx ingress-nginx-controller-599c66746c-g2rmw
- 日志报错解决
报错：Failed to list *v1beta1.Ingress: ingresses.networking.k8s.io is forbidden: User “system:serviceaccount:ingress-nginx:nginx-ingress-serviceaccount” cannot list resource “ingresses” in API group “networking.k8s.io” at the cluster scope
解决：编辑ingress的 mandatory.yaml 在ClusterRole位置添加下述代码然后重新apply -f 即可
- apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses verbs: - list - watch
- 报错2
报错：Failed to update lock: configmaps “ingress-controller-leader” is forbidden: User “system:serviceaccount:ingress-nginx:nginx-ingress-serviceaccount” cannot update resource “configmaps” in API group “” in the namespace “ingress-nginx”
解决：编辑ingress的 mandatory.yaml 在Role位置添加下述代码然后重新apply -f 即可
- apiGroups: - "" resources: - configmaps verbs: - update
- 正确日志输出
[root@k8s nginx-ingress]# kubectl logs -n ingress-nginx ingress-nginx-controller-599c66746c-g2rmw
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 1.1.0
Build: 843a16a8
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.21.5
-------------------------------------------------------------------------------
W0920 05:48:25.803155 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0920 05:48:25.894694 1 main.go:223] "Creating API client" host="https://10.10.0.1:443"
I0920 05:48:26.606925 1 main.go:267] "Running in Kubernetes cluster" major="1" minor="24" git="v1.24.4" state="clean" commit="95ee5ab382d64cfe6c28967f36b53970b8374491" platform="linux/amd64"
I0920 05:48:27.275437 1 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0920 05:48:28.096131 1 ssl.go:531] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0920 05:48:28.363220 1 nginx.go:255] "Starting NGINX Ingress controller"
I0920 05:48:28.510903 1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"6b2a0e79-65f5-43b8-b05b-ad6c5d068a77", APIVersion:"v1", ResourceVersion:"11568", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0920 05:48:30.447454 1 nginx.go:297] "Starting NGINX process"
I0920 05:48:30.455756 1 leaderelection.go:248] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0920 05:48:30.455860 1 nginx.go:317] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0920 05:48:30.461818 1 controller.go:155] "Configuration changes detected, backend reload required"
I0920 05:48:30.539493 1 status.go:84] "New leader elected" identity="nginx-ingress-controller-64cc4648b8-m8pxs"
I0920 05:48:33.551937 1 controller.go:172] "Backend successfully reloaded"
I0920 05:48:33.551996 1 controller.go:183] "Initial sync, sleeping for 1 second"
I0920 05:48:33.553113 1 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-599c66746c-g2rmw", UID:"7413a1d0-e3b7-4a1e-99d2-8e704a9d2ae3", APIVersion:"v1", ResourceVersion:"11935", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
W0920 05:48:34.566596 1 controller.go:201] Dynamic reconfiguration failed: Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
E0920 05:48:34.566632 1 controller.go:205] Unexpected failure reconfiguring NGINX:
Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
E0920 05:48:34.566698 1 queue.go:130] "requeuing" err="Post \"http://127.0.0.1:10246/configuration/backends\": dial tcp 127.0.0.1:10246: connect: connection refused" key="initial-sync"
I0920 05:48:34.566826 1 controller.go:155] "Configuration changes detected, backend reload required"
I0920 05:48:34.723516 1 controller.go:172] "Backend successfully reloaded"
I0920 05:48:34.723679 1 controller.go:183] "Initial sync, sleeping for 1 second"
I0920 05:48:34.761308 1 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-599c66746c-g2rmw", UID:"7413a1d0-e3b7-4a1e-99d2-8e704a9d2ae3", APIVersion:"v1", ResourceVersion:"11935", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0920 05:49:09.840584 1 status.go:84] "New leader elected" identity="ingress-nginx-controller-599c66746c-g2rmw"
I0920 05:49:09.841730 1 leaderelection.go:258] successfully acquired lease ingress-nginx/ingress-controller-leader
- 测试使用
[root@k8s nginx-ingress]# kubectl create deploy echoserver --image=cilium/echoserver --replicas=2
deployment.apps/echoserver created
[root@k8s nginx-ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
echoserver-8585bfb456-75jps 1/1 Running 0 3m52s
echoserver-8585bfb456-scnld 1/1 Running 0 3m53s
[root@k8s nginx-ingress]# kubectl expose deployment echoserver --port=80
service/echoserver exposed
[root@k8s nginx-ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
echoserver ClusterIP 10.10.197.110 80/TCP 38s
kubernetes ClusterIP 10.10.0.1 443/TCP 165m
#正常情况会随机显示两个主机名
[root@k8s nginx-ingress]# curl http://10.10.197.110
Hostname: echoserver-8585bfb456-scnld
Pod Information:
-no pod information available-
Server values:
server_version=nginx: 1.13.3 - lua: 10008
Request Information:
client_address=::ffff:10.122.0.0
method=GET
real path=/
query=
request_version=1.1
request_scheme=http
request_uri=http://10.10.197.110:80/
Request Headers:
accept=*/*
host=10.10.197.110
user-agent=curl/7.29.0
Request Body:
-no body in request-
[root@k8s nginx-ingress]# curl http://10.10.197.110
Hostname: echoserver-8585bfb456-75jps
Pod Information:
-no pod information available-
Server values:
server_version=nginx: 1.13.3 - lua: 10008
Request Information:
client_address=::ffff:10.122.0.0
method=GET
real path=/
query=
request_version=1.1
request_scheme=http
request_uri=http://10.10.197.110:80/
Request Headers:
accept=*/*
host=10.10.197.110
user-agent=curl/7.29.0
Request Body:
-no body in request-
- 测试nodeport
[root@k8s nginx-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.10.68.162 80:31855/TCP,443:31968/TCP 23m
ingress-nginx-controller-admission ClusterIP 10.10.229.71 443/TCP 23m
开启nginx-ingress NodePort端口
[root@k8s nginx-ingress]# kubectl edit svc ingress-nginx-controller -n ingress-nginx
service/ingress-nginx-controller edited
#type改为NodePort
[root@k8s nginx-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.10.68.162 80:31855/TCP,443:31968/TCP 25m
ingress-nginx-controller-admission ClusterIP 10.10.229.71 443/TCP 25m
[root@k8s nginx-ingress]# curl http://192.168.47.50:31855
404 Not Found<<span class="token operator">/</span>title><<span class="token operator">/</span>head>
<body>
<center><h1>404 Not Found<<span class="token operator">/</span>h1><<span class="token operator">/</span>center>
<hr><center>nginx<<span class="token operator">/</span>center>
<<span class="token operator">/</span>body>
<<span class="token operator">/</span>html>
配置ingress策略
<span class="token namespace">[root@k8s nginx-ingress]</span><span class="token comment"># vim ingress-echoserver-test.yaml </span>
<span class="token namespace">[root@k8s nginx-ingress]</span><span class="token comment"># cat ingress-echoserver-test.yaml </span>
apiVersion: networking<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1
kind: Ingress
metadata:
name: ingress-echoserver-test
spec:
rules:
<span class="token operator">-</span> host: echoserver<span class="token punctuation">.</span>test
http:
paths:
<span class="token operator">-</span> path: <span class="token operator">/</span>
pathType: Prefix
backend:
service:
name: echoserver
port:
number: 80
ingressClassName: nginx
<span class="token namespace">[root@k8s nginx-ingress]</span><span class="token comment"># kubectl apply -f ingress-echoserver-test.yaml</span>
ingress<span class="token punctuation">.</span>networking<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/ingress-echoserver-test created
<span class="token namespace">[root@k8s nginx-ingress]</span><span class="token comment"># kubectl get ingress</span>
NAME <span class="token keyword">CLASS</span> HOSTS ADDRESS PORTS AGE
ingress-echoserver-test nginx echoserver<span class="token punctuation">.</span>test 10<span class="token punctuation">.</span>10<span class="token punctuation">.</span>68<span class="token punctuation">.</span>162 80 91s
<span class="token namespace">[root@k8s nginx-ingress]</span><span class="token comment"># curl -H "Host: echoserver.test" http://192.168.47.50:31855</span>
Hostname: echoserver-8585bfb456-75jps
Pod Information:
<span class="token operator">-</span>no pod information available-
Server values:
server_version=nginx: 1<span class="token punctuation">.</span>13<span class="token punctuation">.</span>3 <span class="token operator">-</span> lua: 10008
Request Information:
client_address=::ffff:10<span class="token punctuation">.</span>122<span class="token punctuation">.</span>1<span class="token punctuation">.</span>8
method=GET
real path=<span class="token operator">/</span>
query=
request_version=1<span class="token punctuation">.</span>1
request_scheme=http
request_uri=http:<span class="token operator">/</span><span class="token operator">/</span>echoserver<span class="token punctuation">.</span>test:80/
Request Headers:
accept=<span class="token operator">*</span><span class="token operator">/</span><span class="token operator">*</span>
host=echoserver<span class="token punctuation">.</span>test
user-agent=curl/7<span class="token punctuation">.</span>29<span class="token punctuation">.</span>0
x-forwarded-<span class="token keyword">for</span>=10<span class="token punctuation">.</span>122<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0
x-forwarded-host=echoserver<span class="token punctuation">.</span>test
x-forwarded-port=80
x-forwarded-proto=http
x-forwarded-scheme=http
x-real-ip=10<span class="token punctuation">.</span>122<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0
x-<span class="token function">request-id</span>=57aa1432d2373b71fc966b7f08a01176
x-scheme=http
Request Body:
<span class="token operator">-</span>no body in request-
</code></pre>
<p><mark>到此k8s集群就搭建完成了，后续会有整个cicd流程的部署</mark></p>
<p><mark>如果有问题，可以直接评论区，我会很快回复的</mark></p>
</div>
</div>
</div>
</div>
</div>
<!--PC和WAP自适应版-->
<div id="SOHUCS" sid="1579766775162159104"></div>
<script type="text/javascript" src="/views/front/js/chanyan.js"></script>
<!-- 文章页-底部 动态广告位 -->
<div class="youdao-fixed-ad" id="detail_ad_bottom"></div>
</div>
<div class="col-md-3">
<div class="row" id="ad">
<!-- 文章页-右侧1 动态广告位 -->
<div id="right-1" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_1"> </div>
</div>
<!-- 文章页-右侧2 动态广告位 -->
<div id="right-2" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_2"></div>
</div>
<!-- 文章页-右侧3 动态广告位 -->
<div id="right-3" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_3"></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="container">
<h4 class="pt20 mb15 mt0 border-top">你可能感兴趣的:(linux运维（所有）,云计算,kubernetes,docker)</h4>
<div id="paradigm-article-related">
<div class="recommend-post mb30">
<ul class="widget-links">
<li><a href="/article/1950232820773351424.htm"
title="移动端城市区县二级联动选择功能实现包" target="_blank">移动端城市区县二级联动选择功能实现包</a>
<span class="text-muted">good2know</span>
<div>本文还有配套的精品资源，点击获取简介：本项目是一套为移动端设计的jQuery实现方案，用于简化用户在选择城市和区县时的流程。它包括所有必需文件：HTML、JavaScript、CSS及图片资源。通过动态更新下拉菜单选项，实现城市到区县的联动效果，支持数据异步加载。开发者可以轻松集成此功能到移动网站或应用，并可基于需求进行扩展和优化。1.jQuery移动端解决方案概述jQuery技术简介jQuery</div>
</li>
<li><a href="/article/1950232275262173184.htm"
title="月光下的罪恶（5）" target="_blank">月光下的罪恶（5）</a>
<span class="text-muted">允歌玖沐</span>
<div>5.被孤立顾纨是转校过来的，进入学校后，回头率很高“诶诶诶，你看那女生，哪个系的？”“不知道没见过。”“看那样，一看就是个胆小的货。”顾纨当做没听到，更狠的话她都听过，更何况女生们耍心眼？“他爸爸是做黑生意的，估计女儿也不是什么好的，你以后离他一家子远点。”她走向自己要上课的教室，一进门，所有人的目光看向她，顾纨若无其事的走进教室，开始上课。下课，一群人站起来，但是很显然，她周围的一圈人都不愿意和</div>
</li>
<li><a href="/article/1950232190038110208.htm"
title="day15｜前端框架学习和算法" target="_blank">day15｜前端框架学习和算法</a>
<span class="text-muted">universe_01</span>
<a class="tag" taget="_blank" href="/search/%E5%89%8D%E7%AB%AF/1.htm">前端</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/%E7%AC%94%E8%AE%B0/1.htm">笔记</a>
<div>T22括号生成先把所有情况都画出来，然后（在满足什么情况下）把不符合条件的删除。T78子集要画树状图，把思路清晰。可以用暴力法、回溯法和DFS做这个题DFS深度搜索：每个边都走完，再回溯应用：二叉树搜索，图搜索回溯算法=DFS+剪枝T200岛屿数量（非常经典BFS宽度把树状转化成队列形式，lambda匿名函数“一次性的小函数，没有名字”setup语法糖：让代码更简洁好写的语法ref创建：基本类型的</div>
</li>
<li><a href="/article/1950231640819167232.htm"
title="贝多芬诞辰250周年纪念" target="_blank">贝多芬诞辰250周年纪念</a>
<span class="text-muted">万千星河赴远方</span>
<div>就算不是古典音乐爱好者，你也一定听说过贝多芬。作为古典音乐史上最伟大的音乐家之一，他不仅是古典主义风格的集大成者，同时也是浪漫主义风格的开创者。贝多芬肖像画（1813年）贝多芬的一生共创作了9部交响曲、36首钢琴奏鸣曲、10部小提琴奏鸣曲、16首弦乐四重奏、1部歌剧及2部弥撒曲等等。数量虽然不及前辈海顿、莫扎特多，但他几乎改造了当时所有的音乐表达形式，赋予了它们全新的价值，对后世音乐的发展产生了极</div>
</li>
<li><a href="/article/1950231308781285376.htm"
title="力扣热题100-------54. 螺旋矩阵" target="_blank">力扣热题100-------54. 螺旋矩阵</a>
<span class="text-muted">海航Java之路</span>
<a class="tag" taget="_blank" href="/search/%E5%8A%9B%E6%89%A3/1.htm">力扣</a><a class="tag" taget="_blank" href="/search/leetcode/1.htm">leetcode</a><a class="tag" taget="_blank" href="/search/%E7%9F%A9%E9%98%B5/1.htm">矩阵</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>给你一个m行n列的矩阵matrix，请按照顺时针螺旋顺序，返回矩阵中的所有元素。示例1：输入：matrix=[[1,2,3],[4,5,6],[7,8,9]]输出：[1,2,3,6,9,8,7,4,5]示例2：输入：matrix=[[1,2,3,4],[5,6,7,8],[9,10,11,12]]输出：[1,2,3,4,8,12,11,10,9,5,6,7]提示：m==matrix.lengthn</div>
</li>
<li><a href="/article/1950230678696161280.htm"
title="C++ 计数排序、归并排序、快速排序" target="_blank">C++ 计数排序、归并排序、快速排序</a>
<span class="text-muted">每天搬一点点砖</span>
<a class="tag" taget="_blank" href="/search/c%2B%2B/1.htm">c++</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84/1.htm">数据结构</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a>
<div>计数排序：是一种基于哈希的排序算法。他的基本思想是通过统计每个元素的出现次数，然后根据统计结果将元素依次放入排序后的序列中。这种排序算法适用于范围较小的情况，例如整数范围在0到k之间计数排序步骤：1初始化一个长度为最大元素值加1的计数数组，所有元素初始化为02遍历原始数组，将每个元素值作为索引，在计数数组中对应位置加13将数组清空4遍历计数器数组，按照数组中的元素个数放回到元数组中计数排序的优点和</div>
</li>
<li><a href="/article/1950214458265628672.htm"
title="心力践行营十二期一阶学习打卡" target="_blank">心力践行营十二期一阶学习打卡</a>
<span class="text-muted">LX_王彤彤</span>
<div>姓名：王彤彤时间：2021年4月24日一：朗读师父的十大人生哲学二：师父的早安分享感悟很喜欢这句话：所有的行动都是基于目标的尝试，没有所谓的失败，只是不同尝试后得到的不同结果，让我们更好地调整下一次的行动。三：感恩日记1.我太幸福了，我很感恩姑姑，因为姑姑放假又投喂了我，还给我带了饺子回家，这让我感觉很幸福。谢谢，谢谢，谢谢。2.我太幸福了，我很感恩师父晚上的直播，因为听他的分享我知道怎么更好的去</div>
</li>
<li><a href="/article/1950213690397618176.htm"
title="【阳光️️☀️苑长 幸福365】7月9日《就此刻确知》" target="_blank">【阳光️️☀️苑长 幸福365】7月9日《就此刻确知》</a>
<span class="text-muted">阳光苑长</span>
<div>音频文字原创|阳光苑长图片发自App一夜的梦境，情景相当的壮烈，那是前前后后多个情节的叠加，那是因为极其在意在正义的，完满的，在早已经确认——万古之先已经确认了一切美好的事物当中。当有假象，当有一些不好的势力，试图想去破坏夺取的时候，在梦境中一直得力的征战，为着身边最挚爱的朋友，为他，为所有美好的事业而守望。守望，英文叫做watcher，就是观望者。图片发自App在这个毫不奇怪，事实上一直发生的征</div>
</li>
<li><a href="/article/1950212290682875904.htm"
title="人性四条" target="_blank">人性四条</a>
<span class="text-muted">37f656f2331b</span>
<div>第一条：你想给别人讲道理，你就要混的比他好，让别人闭嘴的从来都不是道理，而是身份。第二条：不花钱就想解决问题的人永远都离不开底层的囚笼，很多时候我们需要的不是一碗鸡汤，而是一个巴掌，巴掌越多，就越管用。第三条：所有的人脉都是假象，想与高层次的人建立人脉，只有三种方式，你具备同等的财富，你具备别人需要的能力和资源，你具备同频的智慧，除此之外，皆是假象。第四条：再好的朋友帮忙，都要记得送礼或是请客吃饭</div>
</li>
<li><a href="/article/1950206309123026944.htm"
title="语文教学反思 ——一单元测试" target="_blank">语文教学反思 ——一单元测试</a>
<span class="text-muted">一抹_绿茶香</span>
<div>我喜欢上语文课，现在最开心的时刻也就是课上那45分钟了。它可以让我和孩子们骑上骏马驰骋在知识的草原上，可以让我们乘着巨轮在书籍的海洋里任意航行……周三举行了一单元测试，今晚一单元的所有内容暂时告一段落。对于这单元我有如下思考：本单元的主题词是“读书”，几篇课文都是围绕着读书来编排的。里面有讲读书乐趣的，讲读书方法的，还有孩子们第一次接触的访谈录等。微笑班级从一年级下学期就开始阅读“闲书”，所以教学</div>
</li>
<li><a href="/article/1950205081676738560.htm"
title="K8s常用的命令" target="_blank">K8s常用的命令</a>
<span class="text-muted">尚未来-</span>
<a class="tag" taget="_blank" href="/search/%E8%BF%90%E7%BB%B4/1.htm">运维</a><a class="tag" taget="_blank" href="/search/k8s/1.htm">k8s</a>
<div>一、基础命令查看集群信息bashkubectlcluster-info#显示集群端点和服务信息查看节点bashkubectlgetnodes#列出所有节点kubectldescribenode#查看节点详细信息查看命名空间bashkubectlgetnamespaces#列出所有命名空间切换命名空间bashkubectlconfigset-context--current--namespace=二</div>
</li>
<li><a href="/article/1950205032758571008.htm"
title="51晨间日记" target="_blank">51晨间日记</a>
<span class="text-muted">讨喜的鱼</span>
<div>2018年11月15日，马上一个月又过去一半，离过年也不远。每到这个时候，心底总有些焦虑。是不是又像去年一样的场景，没赚什么钱，还是一无所有。再有没有以前的潇洒，一无所有，一身轻松，身上有些东西一旦背负，就卸不掉。今天比预计的时间，晚了两个小时，原来说服自己是这么简单的事，告诉自己多睡10分钟。一睁眼，已经过了2小时，这是常有的事。自律必须时刻警惕，而懒惰是要打个盹。工作的话，最近还算顺利。天天写</div>
</li>
<li><a href="/article/1950201927899869184.htm"
title="vllm本地台式机运行(3070显存8G)" target="_blank">vllm本地台式机运行(3070显存8G)</a>
<span class="text-muted">名明鸣冥</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/Qwen/1.htm">Qwen</a><a class="tag" taget="_blank" href="/search/vllm/1.htm">vllm</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E9%83%A8%E7%BD%B2/1.htm">大模型部署</a>
<div>ollama和vllm的对比这块对比网上都很多资料了，这边使用上简单感觉就是ollama很方便部署,占用资料更少,但后续性能会差点,各选项也不一样.ollama安装和使用和docker很像,就不赘述了,这里是看vllm的安装硬件情况介绍30708G显存+-------------------------------------------------------------------------</div>
</li>
<li><a href="/article/1950200714185732096.htm"
title="父母别做“包工头”，让孩子做“小主人”" target="_blank">父母别做“包工头”，让孩子做“小主人”</a>
<span class="text-muted">静云妈妈</span>
<div>文/静云妈妈很多父母，特别是爷爷奶奶外公外婆，俨然一个“包工头”，比如帮孩子穿衣、帮孩子喂饭、帮孩子洗漱、帮孩子处理与其他小朋友发生的冲突等等。这对孩子并不好，其实只是我们打着“爱孩子”的名义，剥夺了孩子自我发展的权利。像教孩子走路一样不会有哪位家长打算抱一个正常的孩子一辈子，我们总是在孩子适合的年龄想各种办法辅助孩子自己行走，最终孩子由摇摇摆摆到走得十分平顺，甚至跑步前进。面对孩子所有的事情，家</div>
</li>
<li><a href="/article/1950199279641489408.htm"
title="Android通知(Notification)全面解析：从基础到高级应用" target="_blank">Android通知(Notification)全面解析：从基础到高级应用</a>
<span class="text-muted"></span>
<div>一、Android通知概述通知(Notification)是Android系统中用于在应用之外向用户传递信息的重要机制。当应用需要告知用户某些事件或信息时，可以通过通知在状态栏显示图标，用户下拉通知栏即可查看详细信息。这种机制几乎被所有现代应用采用，用于推送新闻、消息、广告等内容3。与Toast相比，Notification的优势在于：可以长时间停留在通知栏，适合内容较多且需要持久展示的信息支持丰</div>
</li>
<li><a href="/article/1950198307473125376.htm"
title="个人创业能月入两万的小生意，适合新手做的小生意项目！" target="_blank">个人创业能月入两万的小生意，适合新手做的小生意项目！</a>
<span class="text-muted">优惠券高省</span>
<div>互联网浪潮的风起云涌，使得许多人萌生了创业的想法，但同时创业也并不简单，本身市场里面的水很深，如果大家初次创业的话，很有可能会遭受经济的损失。小编为大家着重介绍适合所有人做的月入2万的10个小生意，一起来看看吧。首先推荐大家做互联网电商创业，零投资，轻创业，没有压力，上手就赚钱！门槛低，天花板高！高省时代在变，分享经济是趋势！什么是分享经济：把你认为好的告诉身边的人叫分享，，所以分享经济就是把好的</div>
</li>
<li><a href="/article/1950197670534508544.htm"
title="舌尖记忆" target="_blank">舌尖记忆</a>
<span class="text-muted">Y十一吖</span>
<div>“来来来，大家尝尝我网购的老月饼。五仁馅的，看看正不正宗。”9月16日下午，刚进办公室的刘队长就热情地招呼大家吃月饼。“这个太甜了，还是小时候家里做的香。”70后的周师说起小时候自己加工的月饼，一时勾起了我儿时的记忆。那个时候，油田家属区还不在城市里。往往都是工作在哪里，家就在哪里。周围的邻居都是来自天南地北。过节也不像现在，去一趟超市，所有物资都能准备齐全，往往要提前做准备。印象最深刻的，便是中</div>
</li>
<li><a href="/article/1950195246310682624.htm"
title="【日更挑战】本季联盟50大球星（46-44）" target="_blank">【日更挑战】本季联盟50大球星（46-44）</a>
<span class="text-muted">扁圆柱体</span>
<div>日更挑战当前排名：第153天，第2269名，较昨日前进10名。46.KrisLetang，后卫，匹兹堡企鹅上赛季，Letang在55场比赛中得到45分（7球，38助攻），在所有后卫中，和闪电的VictorHedman并列第三，他在同等人数下的净胜球是+20，位列全队第一。而这个成绩在联盟自2009-10赛季统计该项指标后，是个人第二好成绩（+22,2011-12赛季）。Letang平均每场有24分</div>
</li>
<li><a href="/article/1950192091439099904.htm"
title="K8S 常用命令全解析：高效管理容器化集群" target="_blank">K8S 常用命令全解析：高效管理容器化集群</a>
<span class="text-muted">恩爸编程</span>
<a class="tag" taget="_blank" href="/search/docker/1.htm">docker</a><a class="tag" taget="_blank" href="/search/kubernetes/1.htm">kubernetes</a><a class="tag" taget="_blank" href="/search/%E5%AE%B9%E5%99%A8/1.htm">容器</a><a class="tag" taget="_blank" href="/search/k8s%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/1.htm">k8s常用命令</a><a class="tag" taget="_blank" href="/search/k8s%E6%9C%89%E5%93%AA%E4%BA%9B%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/1.htm">k8s有哪些常用命令</a><a class="tag" taget="_blank" href="/search/k8s%E5%91%BD%E4%BB%A4%E6%9C%89%E5%93%AA%E4%BA%9B/1.htm">k8s命令有哪些</a><a class="tag" taget="_blank" href="/search/K8S%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4%E6%9C%89%E5%93%AA%E4%BA%9B/1.htm">K8S常用命令有哪些</a>
<div>K8S常用命令全解析：高效管理容器化集群一、引言Kubernetes（K8S）作为强大的容器编排平台，其丰富的命令行工具（kubectl）为用户提供了便捷的方式来管理集群中的各种资源。熟练掌握K8S常用命令对于开发人员和运维人员至关重要，能够有效提高容器化应用的部署、监控与维护效率。本文将详细介绍一些K8S常用命令及其使用案例。二、基础资源操作命令（一）kubectlcreate功能：用于创建K8</div>
</li>
<li><a href="/article/1950190326673108992.htm"
title="Spark SQL架构及高级用法" target="_blank">Spark SQL架构及高级用法</a>
<span class="text-muted">Aurora_NeAr</span>
<a class="tag" taget="_blank" href="/search/spark/1.htm">spark</a><a class="tag" taget="_blank" href="/search/sql/1.htm">sql</a><a class="tag" taget="_blank" href="/search/%E6%9E%B6%E6%9E%84/1.htm">架构</a>
<div>SparkSQL架构概述架构核心组件API层（用户接口）输入方式：SQL查询；DataFrame/DatasetAPI。统一性：所有接口最终转换为逻辑计划树（LogicalPlan），进入优化流程。编译器层（Catalyst优化器）核心引擎：基于规则的优化器（Rule-BasedOptimizer,RBO）与成本优化器（Cost-BasedOptimizer,CBO）。处理流程：阶段输入输出关键动</div>
</li>
<li><a href="/article/1950190147333058560.htm"
title="时间是个神" target="_blank">时间是个神</a>
<span class="text-muted">王家二少2</span>
<div>很多时候，我们站在原地，以为总会等到那个人。可是时光荏苒，命运就像不停发车的车站，直到某一天，我们才顿然了解，所有的一切都不会再来了。我们只是因为幼稚或单纯或其他的原因，上晚了车。我们所珍惜的，不一定是别人所留念的。每一段关系都有一个保质期，有的或许如钻石般恒远，有的却只如钢铁般，看似坚牢，却终会腐蚀、生锈，最后被其他的东西所替代。——当我我回头找你，却发现早已与你背道而驰。</div>
</li>
<li><a href="/article/1950188184285540352.htm"
title="Mac OSX 下的mysql数据库文件存放位置" target="_blank">Mac OSX 下的mysql数据库文件存放位置</a>
<span class="text-muted">Bruuuces</span>
<a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a><a class="tag" taget="_blank" href="/search/mac/1.htm">mac</a><a class="tag" taget="_blank" href="/search/osx/1.htm">osx</a><a class="tag" taget="_blank" href="/search/%E4%BD%8D%E7%BD%AE/1.htm">位置</a><a class="tag" taget="_blank" href="/search/%E5%AD%98%E6%94%BE/1.htm">存放</a>
<div>之前我的mysql的系统数据库里的表被我玩坏了，万般无奈之下只得删除所有mysql的东西重新构建数据库。按照网上搜到的内容删除后重装发现数据库没有什么变化。于是自己在每个可能存放数据库文件的目录查找，最终确认目录位置如下:使用HomeBrew安装为/usr/local/var/mysql使用官方下载的dmg镜像安装为/usr/local/mysql删除这个目录再重新安装mysql就会重新生成系统数</div>
</li>
<li><a href="/article/1950185188122226688.htm"
title="禁止缠爱宋知闲江窈(禁止缠爱小说)全文免费阅读无弹窗" target="_blank">禁止缠爱宋知闲江窈(禁止缠爱小说)全文免费阅读无弹窗</a>
<span class="text-muted">海边书楼</span>
<div>禁止缠爱宋知闲江窈(禁止缠爱小说)全文免费阅读无弹窗主角：宋知闲江窈简介：飞机上一场阴差阳错的情事过后，宋知闲对江窈的肉体食髓知味。多夜欢好后。她问，“娶我？”宋知闲淡漠，“不娶，没兴趣。”再后来，那个永远冷漠的高岭之花在所有人面前，死死拽着她的手，红着眼，“江窈，我求你别走！”江窈微微一笑，心如止水。“放手，明天我嫁人。”可关注微信公众号【寒风书楼】去回个书号【307】，即可免费阅读【禁止缠爱】</div>
</li>
<li><a href="/article/1950185059596169216.htm"
title="觉得做人累了就进来看看这些！" target="_blank">觉得做人累了就进来看看这些！</a>
<span class="text-muted">影帝黄博</span>
<div>学会沉默有时候，你被人误解，你不想争辩，所以选择沈默。本来就不是所有的人都得了解你，因此你认为不必对全世界喊话。却也有时候，你被最爱的人误解，你难过到不想争辩，也只有选择沈默。全世界都可以不懂你，但他应该懂，若他竟然不能懂，还有什么话可说?生命中往往有连舒伯特都无言以对的时刻，毕竟不是所有的是非都能条列清楚，甚至可能根本没有真正的是与非。那么，不想说话，就不说吧，在多说无益的时候，也许沈默就是最好</div>
</li>
<li><a href="/article/1950184804079169536.htm"
title="思维导图中的3A" target="_blank">思维导图中的3A</a>
<span class="text-muted">画画的小常</span>
<div>图片发自App在学习思维导图3A之前，我们先来看看东方古国传统教书过程。分三步：听话，合作，变化。听话，学生摹仿老师，在这个过程中除非必需问题需要老师澄清解释，其他所有的都要求先行记忆。合作，这一阶段大多已经掌握基本知识点，允许学生提出问题，老师引导学生采用合适的方法，让学生亲自动手，解决问题。变化，经历了前面2个阶段，知识已经彻底掌握，此阶段要充分发挥洞察力和生发出新的思想以回馈老师。较之传统教</div>
</li>
<li><a href="/article/1950183520588591104.htm"
title="搭建云手机教程" target="_blank">搭建云手机教程</a>
<span class="text-muted">云博客-资源宝</span>
<a class="tag" taget="_blank" href="/search/%E6%99%BA%E8%83%BD%E6%89%8B%E6%9C%BA/1.htm">智能手机</a>
<div>搭建云手机教程本教程由分享：ziyouhua资源宝整理分享：www.httple.net首先检查自己vps是否支持这个项目sudoaptinstallcpu-checkerkvm-ok如果显示INFO:/dev/kvmexistsKVMaccelerationcanbeused表示支持，可以继续往下看ac13首先安装dockercurl-fsSLhttps://get.docker.com|sud</div>
</li>
<li><a href="/article/1950177414323695616.htm"
title="漫画人家隐私政策" target="_blank">漫画人家隐私政策</a>
<span class="text-muted">呵呵一笑_e8f9</span>
<div>漫画人家APP（以下简称漫画人家）非常重视对您的个人隐私保护，有时候我们需要某些信息才能为您提供您请求的服务，本隐私声明解释了这些情况下的数据收集和使用情况。本隐私声明适用于漫画人家的所有相关服务，随着漫画人家服务范围的扩大，隐私声明的内容可由漫画人家随时更新，且毋须另行通知。更新后的隐私声明一旦在网页上公布即有效代替原来的隐私声明。我们收集哪些信息通常，您能在匿名的状态下访问漫画人家并获取信息。</div>
</li>
<li><a href="/article/1950176839670493184.htm"
title="全局修改GitLab14默认语言为中文" target="_blank">全局修改GitLab14默认语言为中文</a>
<span class="text-muted"></span>
<div>GitLab安装成功后默认语言是英语，只有登录后才能手动指定为中文，且这个配置只对自己生效，经查阅资料后，总结全局修改GitLab14默认语言为中文方法如下：0.进入容器如果你用Docker部署的GitLab，那么需要使用命令sudodockerexec-itgitlab/bin/bash进入容器1.修改rails配置文件打开/opt/gitlab/embedded/service/gitlab-</div>
</li>
<li><a href="/article/1950173353989959680.htm"
title="新华小学的人还在上学" target="_blank">新华小学的人还在上学</a>
<span class="text-muted">快乐_c0d8</span>
<div>今天新华小学的所有人还在上学，是因为他们的诗都考完了。他们今天下午又上学了，我很开心，因为他们不来，他们就不能惹事儿了，所以我很开心。</div>
</li>
<li><a href="/article/1950167758767452160.htm"
title="负载均衡-加权随机算法" target="_blank">负载均衡-加权随机算法</a>
<span class="text-muted">BP白朴</span>
<a class="tag" taget="_blank" href="/search/Nginx/1.htm">Nginx</a><a class="tag" taget="_blank" href="/search/%E8%B4%9F%E8%BD%BD%E5%9D%87%E8%A1%A1/1.htm">负载均衡</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/%E6%9C%8D%E5%8A%A1%E5%99%A8/1.htm">服务器</a>
<div>负载均衡-加权随机算法由于访问概率大致相同，所以如果部分服务器性能不一致的话，容易导致性能差的服务器压力过大，所以要根据服务器性能不一致的情况，给性能好的服务器多处理请求，给差的少分配请求（能者多劳）所以就需要在随机算法的基础上给每台服务器设置权重，延伸为加权随机算法1、将应用服务器集群的IP存到Map里,每个IP对应有一个权重2、创建一个List,来将所有权重下的IP存到list里面如：192.</div>
</li>
<li><a href="/article/60.htm"
title="Dom" target="_blank">Dom</a>
<span class="text-muted">周华华</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a><a class="tag" taget="_blank" href="/search/html/1.htm">html</a>
<div><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml&q</div>
</li>
<li><a href="/article/187.htm"
title="【Spark九十六】RDD API之combineByKey" target="_blank">【Spark九十六】RDD API之combineByKey</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/spark/1.htm">spark</a>
<div>1. combineByKey函数的运行机制
RDD提供了很多针对元素类型为(K,V)的API，这些API封装在PairRDDFunctions类中，通过Scala隐式转换使用。这些API实现上是借助于combineByKey实现的。combineByKey函数本身也是RDD开放给Spark开发人员使用的API之一
首先看一下combineByKey的方法说明：</div>
</li>
<li><a href="/article/314.htm"
title="msyql设置密码报错：ERROR 1372 (HY000): 解决方法详解" target="_blank">msyql设置密码报错：ERROR 1372 (HY000): 解决方法详解</a>
<span class="text-muted">daizj</span>
<a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a><a class="tag" taget="_blank" href="/search/%E8%AE%BE%E7%BD%AE%E5%AF%86%E7%A0%81/1.htm">设置密码</a>
<div>MySql给用户设置权限同时指定访问密码时，会提示如下错误：
ERROR 1372 (HY000): Password hash should be a 41-digit hexadecimal number；
问题原因：你输入的密码是明文。不允许这么输入。
解决办法：用select password('你想输入的密码');查询出你的密码对应的字符串，
然后</div>
</li>
<li><a href="/article/441.htm"
title="路漫漫其修远兮 吾将上下而求索" target="_blank">路漫漫其修远兮 吾将上下而求索</a>
<span class="text-muted">周凡杨</span>
<a class="tag" taget="_blank" href="/search/%E5%AD%A6%E4%B9%A0+%E6%80%9D%E7%B4%A2/1.htm">学习 思索</a>
<div>王国维在他的《人间词话》中曾经概括了为学的三种境界古今之成大事业、大学问者，罔不经过三种之境界。“昨夜西风凋碧树。独上高楼，望尽天涯路。”此第一境界也。“衣带渐宽终不悔，为伊消得人憔悴。”此第二境界也。“众里寻他千百度，蓦然回首，那人却在灯火阑珊处。”此第三境界也。学习技术，这也是你必须经历的三种境界。第一层境界是说，学习的路是漫漫的，你必须做好充分的思想准备，如果半途而废还不如不要开始。这里，注</div>
</li>
<li><a href="/article/568.htm"
title="Hadoop(二)对话单的操作" target="_blank">Hadoop(二)对话单的操作</a>
<span class="text-muted">朱辉辉33</span>
<a class="tag" taget="_blank" href="/search/hadoop/1.htm">hadoop</a>
<div>Debug：
1、
A = LOAD '/user/hue/task.txt' USING PigStorage(' ')
AS (col1,col2,col3);
DUMP A;
//输出结果前几行示例：
(>ggsnPDPRecord(21),,)
(-->recordType(0),,)
(-->networkInitiation(1),,)
</div>
</li>
<li><a href="/article/695.htm"
title="web报表工具FineReport常用函数的用法总结（日期和时间函数）" target="_blank">web报表工具FineReport常用函数的用法总结（日期和时间函数）</a>
<span class="text-muted">老A不折腾</span>
<a class="tag" taget="_blank" href="/search/finereport/1.htm">finereport</a><a class="tag" taget="_blank" href="/search/%E6%8A%A5%E8%A1%A8%E5%B7%A5%E5%85%B7/1.htm">报表工具</a><a class="tag" taget="_blank" href="/search/web%E5%BC%80%E5%8F%91/1.htm">web开发</a>
<div>web报表工具FineReport常用函数的用法总结（日期和时间函数）
说明：凡函数中以日期作为参数因子的，其中日期的形式都必须是yy/mm/dd。而且必须用英文环境下双引号(" ")引用。
DATE
DATE(year,month,day):返回一个表示某一特定日期的系列数。
Year:代表年，可为一到四位数。
Month:代表月份。</div>
</li>
<li><a href="/article/822.htm"
title="c++ 宏定义中的##操作符" target="_blank">c++ 宏定义中的##操作符</a>
<span class="text-muted">墙头上一根草</span>
<a class="tag" taget="_blank" href="/search/C%2B%2B/1.htm">C++</a>
<div>#与##在宏定义中的--宏展开 #include <stdio.h> #define f(a,b) a##b #define g(a) #a #define h(a) g(a) int main() { &nbs</div>
</li>
<li><a href="/article/949.htm"
title="分析Spring源代码之，DI的实现" target="_blank">分析Spring源代码之，DI的实现</a>
<span class="text-muted">aijuans</span>
<a class="tag" taget="_blank" href="/search/spring/1.htm">spring</a><a class="tag" taget="_blank" href="/search/DI/1.htm">DI</a><a class="tag" taget="_blank" href="/search/%E7%8E%B0/1.htm">现</a><a class="tag" taget="_blank" href="/search/%E6%BA%90%E4%BB%A3%E7%A0%81/1.htm">源代码</a>
<div>(转)
分析Spring源代码之，DI的实现
2012/1/3 by tony
接着上次的讲，以下这个sample
[java]
view plain
copy
print
</div>
</li>
<li><a href="/article/1076.htm"
title="for循环的进化" target="_blank">for循环的进化</a>
<span class="text-muted">alxw4616</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a>
<div>// for循环的进化
// 菜鸟
for (var i = 0; i < Things.length ; i++) {
// Things[i]
}
// 老鸟
for (var i = 0, len = Things.length; i < len; i++) {
// Things[i]
}
// 大师
for (var i = Things.le</div>
</li>
<li><a href="/article/1203.htm"
title="网络编程Socket和ServerSocket简单的使用" target="_blank">网络编程Socket和ServerSocket简单的使用</a>
<span class="text-muted">百合不是茶</span>
<a class="tag" taget="_blank" href="/search/%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B%E5%9F%BA%E7%A1%80/1.htm">网络编程基础</a><a class="tag" taget="_blank" href="/search/IP%E5%9C%B0%E5%9D%80%E7%AB%AF%E5%8F%A3/1.htm">IP地址端口</a>
<div>
网络编程;TCP/IP协议
网络:实现计算机之间的信息共享,数据资源的交换
协议:数据交换需要遵守的一种协议,按照约定的数据格式等写出去
端口:用于计算机之间的通信
每运行一个程序，系统会分配一个编号给该程序，作为和外界交换数据的唯一标识
0~65535
查看被使用的</div>
</li>
<li><a href="/article/1330.htm"
title="JDK1.5 生产消费者" target="_blank">JDK1.5 生产消费者</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/thread/1.htm">thread</a><a class="tag" taget="_blank" href="/search/%E7%94%9F%E4%BA%A7%E6%B6%88%E8%B4%B9%E8%80%85/1.htm">生产消费者</a><a class="tag" taget="_blank" href="/search/java%E5%A4%9A%E7%BA%BF%E7%A8%8B/1.htm">java多线程</a>
<div>ArrayBlockingQueue：
一个由数组支持的有界阻塞队列。此队列按 FIFO（先进先出）原则对元素进行排序。队列的头部 是在队列中存在时间最长的元素。队列的尾部 是在队列中存在时间最短的元素。新元素插入到队列的尾部，队列检索操作则是从队列头部开始获得元素。
ArrayBlockingQueue的常用方法：
</div>
</li>
<li><a href="/article/1457.htm"
title="JAVA版身份证获取性别、出生日期及年龄" target="_blank">JAVA版身份证获取性别、出生日期及年龄</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E6%80%A7%E5%88%AB/1.htm">性别</a><a class="tag" taget="_blank" href="/search/%E5%87%BA%E7%94%9F%E6%97%A5%E6%9C%9F/1.htm">出生日期</a><a class="tag" taget="_blank" href="/search/%E5%B9%B4%E9%BE%84/1.htm">年龄</a>
<div> 工作中需要根据身份证获取性别、出生日期及年龄，且要还要支持15位长度的身份证号码，网上搜索了一下，经过测试好像多少存在点问题，干脆自已写一个。
CertificateNo.java
package com.bijian.study;
import java.util.Calendar;
import </div>
</li>
<li><a href="/article/1584.htm"
title="【Java范型六】范型与枚举" target="_blank">【Java范型六】范型与枚举</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>首先，枚举类型的定义不能带有类型参数，所以，不能把枚举类型定义为范型枚举类，例如下面的枚举类定义是有编译错的
public enum EnumGenerics<T> { //编译错，提示枚举不能带有范型参数
OK, ERROR;
public <T> T get(T type) {
return null;
</div>
</li>
<li><a href="/article/1711.htm"
title="【Nginx五】Nginx常用日志格式含义" target="_blank">【Nginx五】Nginx常用日志格式含义</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/nginx/1.htm">nginx</a>
<div>1. log_format
1.1 log_format指令用于指定日志的格式，格式：
log_format name(格式名称) type(格式样式)
1.2 如下是一个常用的Nginx日志格式：
log_format main '[$time_local]|$request_time|$status|$body_bytes</div>
</li>
<li><a href="/article/1838.htm"
title="Lua 语言 15 分钟快速入门" target="_blank">Lua 语言 15 分钟快速入门</a>
<span class="text-muted">ronin47</span>
<a class="tag" taget="_blank" href="/search/lua+%E5%9F%BA%E7%A1%80/1.htm">lua 基础</a>
<div>-
-
单行注释
-
-
[[
[多行注释]
-
-
]]
-
-
-
-
-
-
-
-
-
-
-
1.
变量 & 控制流
-
-
-
-
-
-
-
-
-
-
num
=
23
-
-
数字都是双精度
str
=
'aspythonstring'
</div>
</li>
<li><a href="/article/1965.htm"
title="java-35.求一个矩阵中最大的二维矩阵 ( 元素和最大 )" target="_blank">java-35.求一个矩阵中最大的二维矩阵 ( 元素和最大 )</a>
<span class="text-muted">bylijinnan</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>the idea is from:
http://blog.csdn.net/zhanxinhang/article/details/6731134
public class MaxSubMatrix {
/**see http://blog.csdn.net/zhanxinhang/article/details/6731134
* Q35
求一个矩阵中最大的二维</div>
</li>
<li><a href="/article/2092.htm"
title="mongoDB文档型数据库特点" target="_blank">mongoDB文档型数据库特点</a>
<span class="text-muted">开窍的石头</span>
<a class="tag" taget="_blank" href="/search/mongoDB%E6%96%87%E6%A1%A3%E5%9E%8B%E6%95%B0%E6%8D%AE%E5%BA%93%E7%89%B9%E7%82%B9/1.htm">mongoDB文档型数据库特点</a>
<div>MongoDD: 文档型数据库存储的是Bson文档-->json的二进制
特点：内部是执行引擎是js解释器，把文档转成Bson结构，在查询时转换成js对象。
mongoDB传统型数据库对比
传统类型数据库：结构化数据，定好了表结构后每一个内容符合表结构的。也就是说每一行每一列的数据都是一样的
文档型数据库：不用定好数据结构，</div>
</li>
<li><a href="/article/2219.htm"
title="[毕业季节]欢迎广大毕业生加入JAVA程序员的行列" target="_blank">[毕业季节]欢迎广大毕业生加入JAVA程序员的行列</a>
<span class="text-muted">comsci</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>
一年一度的毕业季来临了。。。。。。。。
正在投简历的学弟学妹们。。。如果觉得学校推荐的单位和公司不适合自己的兴趣和专业，可以考虑来我们软件行业，做一名职业程序员。。。
软件行业的开发工具中，对初学者最友好的就是JAVA语言了，网络上不仅仅有大量的</div>
</li>
<li><a href="/article/2346.htm"
title="PHP操作Excel – PHPExcel 基本用法详解" target="_blank">PHP操作Excel – PHPExcel 基本用法详解</a>
<span class="text-muted">cuiyadll</span>
<a class="tag" taget="_blank" href="/search/PHP/1.htm">PHP</a><a class="tag" taget="_blank" href="/search/Excel/1.htm">Excel</a>
<div>导出excel属性设置//Include classrequire_once('Classes/PHPExcel.php');require_once('Classes/PHPExcel/Writer/Excel2007.php');$objPHPExcel = new PHPExcel();//Set properties 设置文件属性$objPHPExcel->getProperties</div>
</li>
<li><a href="/article/2473.htm"
title="IBM Webshpere MQ Client User Issue (MCAUSER)" target="_blank">IBM Webshpere MQ Client User Issue (MCAUSER)</a>
<span class="text-muted">darrenzhu</span>
<a class="tag" taget="_blank" href="/search/IBM/1.htm">IBM</a><a class="tag" taget="_blank" href="/search/jms/1.htm">jms</a><a class="tag" taget="_blank" href="/search/user/1.htm">user</a><a class="tag" taget="_blank" href="/search/MQ/1.htm">MQ</a><a class="tag" taget="_blank" href="/search/MCAUSER/1.htm">MCAUSER</a>
<div>IBM MQ JMS Client去连接远端MQ Server的时候，需要提供User和Password吗？
答案是根据情况而定，取决于所定义的Channel里面的属性Message channel agent user identifier (MCAUSER)的设置。
http://stackoverflow.com/questions/20209429/how-mca-user-i</div>
</li>
<li><a href="/article/2600.htm"
title="网线的接法" target="_blank">网线的接法</a>
<span class="text-muted">dcj3sjt126com</span>
<div>一、PC连HUB (直连线)A端：（标准568B）：白橙，橙，白绿，蓝，白蓝，绿，白棕，棕。 B端：（标准568B）：白橙，橙，白绿，蓝，白蓝，绿，白棕，棕。 二、PC连PC （交叉线）A端：(568A)： 白绿，绿，白橙，蓝，白蓝，橙，白棕，棕； B端：（标准568B）：白橙，橙，白绿，蓝，白蓝，绿，白棕，棕。 三、HUB连HUB&nb</div>
</li>
<li><a href="/article/2727.htm"
title="Vimium插件让键盘党像操作Vim一样操作Chrome" target="_blank">Vimium插件让键盘党像操作Vim一样操作Chrome</a>
<span class="text-muted">dcj3sjt126com</span>
<a class="tag" taget="_blank" href="/search/chrome/1.htm">chrome</a><a class="tag" taget="_blank" href="/search/vim/1.htm">vim</a>
<div>什么是键盘党？
键盘党是指尽可能将所有电脑操作用键盘来完成，而不去动鼠标的人。鼠标应该说是新手们的最爱，很直观，指哪点哪，很听话！不过常常使用电脑的人，如果一直使用鼠标的话，手会发酸，因为操作鼠标的时候，手臂不是在一个自然的状态，臂肌会处于绷紧状态。而使用键盘则双手是放松状态，只有手指在动。而且尽量少的从鼠标移动到键盘来回操作，也省不少事。
在chrome里安装 vimium 插件
</div>
</li>
<li><a href="/article/2854.htm"
title="MongoDB查询（2）——数组查询[六]" target="_blank">MongoDB查询（2）——数组查询[六]</a>
<span class="text-muted">eksliang</span>
<a class="tag" taget="_blank" href="/search/mongodb/1.htm">mongodb</a><a class="tag" taget="_blank" href="/search/MongoDB%E6%9F%A5%E8%AF%A2%E6%95%B0%E7%BB%84/1.htm">MongoDB查询数组</a>
<div>MongoDB查询数组
转载请出自出处：http://eksliang.iteye.com/blog/2177292 一、概述
MongoDB查询数组与查询标量值是一样的，例如，有一个水果列表，如下所示：
> db.food.find()
{ "_id" : "001", "fruits" : [ "苹</div>
</li>
<li><a href="/article/2981.htm"
title="cordova读写文件（1）" target="_blank">cordova读写文件（1）</a>
<span class="text-muted">gundumw100</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a><a class="tag" taget="_blank" href="/search/Cordova/1.htm">Cordova</a>
<div>使用cordova可以很方便的在手机sdcard中读写文件。
首先需要安装cordova插件：file
命令为：
cordova plugin add org.apache.cordova.file
然后就可以读写文件了，这里我先是写入一个文件，具体的JS代码为：
var datas=null;//datas need write
var directory=&</div>
</li>
<li><a href="/article/3108.htm"
title="HTML5 FormData 进行文件jquery ajax 上传 到又拍云" target="_blank">HTML5 FormData 进行文件jquery ajax 上传 到又拍云</a>
<span class="text-muted">ileson</span>
<a class="tag" taget="_blank" href="/search/jquery/1.htm">jquery</a><a class="tag" taget="_blank" href="/search/Ajax/1.htm">Ajax</a><a class="tag" taget="_blank" href="/search/html5/1.htm">html5</a><a class="tag" taget="_blank" href="/search/FormData/1.htm">FormData</a>
<div>html5 新东西：FormData 可以提交二进制数据。
页面test.html
<!DOCTYPE>
<html>
<head>
<title> formdata file jquery ajax upload</title>
</head>
<body>
<</div>
</li>
<li><a href="/article/3235.htm"
title="swift appearanceWhenContainedIn:(version1.2 xcode6.4)" target="_blank">swift appearanceWhenContainedIn:(version1.2 xcode6.4)</a>
<span class="text-muted">啸笑天</span>
<a class="tag" taget="_blank" href="/search/version/1.htm">version</a>
<div>
swift1.2中没有oc中对应的方法：
+ (instancetype)appearanceWhenContainedIn:(Class <UIAppearanceContainer>)ContainerClass, ... NS_REQUIRES_NIL_TERMINATION;
解决方法：
在swift项目中新建oc类如下：
#import &</div>
</li>
<li><a href="/article/3362.htm"
title="java实现SMTP邮件服务器" target="_blank">java实现SMTP邮件服务器</a>
<span class="text-muted">macroli</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E7%BC%96%E7%A8%8B/1.htm">编程</a>
<div>电子邮件传递可以由多种协议来实现。目前，在Internet 网上最流行的三种电子邮件协议是SMTP、POP3 和 IMAP，下面分别简单介绍。
◆ SMTP 协议
简单邮件传输协议(Simple Mail Transfer Protocol,SMTP)是一个运行在TCP/IP之上的协议，用它发送和接收电子邮件。SMTP 服务器在默认端口25上监听。SMTP客户使用一组简单的、基于文本的</div>
</li>
<li><a href="/article/3489.htm"
title="mongodb group by having where 查询sql" target="_blank">mongodb group by having where 查询sql</a>
<span class="text-muted">qiaolevip</span>
<a class="tag" taget="_blank" href="/search/%E6%AF%8F%E5%A4%A9%E8%BF%9B%E6%AD%A5%E4%B8%80%E7%82%B9%E7%82%B9/1.htm">每天进步一点点</a><a class="tag" taget="_blank" href="/search/%E5%AD%A6%E4%B9%A0%E6%B0%B8%E6%97%A0%E6%AD%A2%E5%A2%83/1.htm">学习永无止境</a><a class="tag" taget="_blank" href="/search/mongo/1.htm">mongo</a><a class="tag" taget="_blank" href="/search/%E7%BA%B5%E8%A7%82%E5%8D%83%E8%B1%A1/1.htm">纵观千象</a>
<div>SELECT cust_id,
SUM(price) as total
FROM orders
WHERE status = 'A'
GROUP BY cust_id
HAVING total > 250
db.orders.aggregate( [
{ $match: { status: 'A' } },
{
$group: {
</div>
</li>
<li><a href="/article/3616.htm"
title="Struts2 Pojo（六）" target="_blank">Struts2 Pojo（六）</a>
<span class="text-muted">Luob.</span>
<a class="tag" taget="_blank" href="/search/POJO/1.htm">POJO</a><a class="tag" taget="_blank" href="/search/strust2/1.htm">strust2</a>
<div>注意：附件中有完整案例
1.采用POJO对象的方法进行赋值和传值
2.web配置
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5"
xmlns="http://java.sun.com/xml/ns/javaee&q</div>
</li>
<li><a href="/article/3743.htm"
title="struts2步骤" target="_blank">struts2步骤</a>
<span class="text-muted">wuai</span>
<a class="tag" taget="_blank" href="/search/struts/1.htm">struts</a>
<div>1、添加jar包
2、在web.xml中配置过滤器
<filter>
<filter-name>struts2</filter-name>
<filter-class>org.apache.st</div>
</li>
</ul>
</div>
</div>
</div>
<div>
<div class="container">
<div class="indexes">
<strong>按字母分类：</strong>
<a href="/tags/A/1.htm" target="_blank">A</a><a href="/tags/B/1.htm" target="_blank">B</a><a href="/tags/C/1.htm" target="_blank">C</a><a
href="/tags/D/1.htm" target="_blank">D</a><a href="/tags/E/1.htm" target="_blank">E</a><a href="/tags/F/1.htm" target="_blank">F</a><a
href="/tags/G/1.htm" target="_blank">G</a><a href="/tags/H/1.htm" target="_blank">H</a><a href="/tags/I/1.htm" target="_blank">I</a><a
href="/tags/J/1.htm" target="_blank">J</a><a href="/tags/K/1.htm" target="_blank">K</a><a href="/tags/L/1.htm" target="_blank">L</a><a
href="/tags/M/1.htm" target="_blank">M</a><a href="/tags/N/1.htm" target="_blank">N</a><a href="/tags/O/1.htm" target="_blank">O</a><a
href="/tags/P/1.htm" target="_blank">P</a><a href="/tags/Q/1.htm" target="_blank">Q</a><a href="/tags/R/1.htm" target="_blank">R</a><a
href="/tags/S/1.htm" target="_blank">S</a><a href="/tags/T/1.htm" target="_blank">T</a><a href="/tags/U/1.htm" target="_blank">U</a><a
href="/tags/V/1.htm" target="_blank">V</a><a href="/tags/W/1.htm" target="_blank">W</a><a href="/tags/X/1.htm" target="_blank">X</a><a
href="/tags/Y/1.htm" target="_blank">Y</a><a href="/tags/Z/1.htm" target="_blank">Z</a><a href="/tags/0/1.htm" target="_blank">其他</a>
</div>
</div>
</div>
<footer id="footer" class="mb30 mt30">
<div class="container">
<div class="footBglm">
<a target="_blank" href="/">首页</a> -
<a target="_blank" href="/custom/about.htm">关于我们</a> -
<a target="_blank" href="/search/Java/1.htm">站内搜索</a> -
<a target="_blank" href="/sitemap.txt">Sitemap</a> -
<a target="_blank" href="/custom/delete.htm">侵权投诉</a>
</div>
<div class="copyright">版权所有 IT知识库 CopyRight © 2000-2050 E-COM-NET.COM , All Rights Reserved.
<!-- <a href="https://beian.miit.gov.cn/" rel="nofollow" target="_blank">京ICP备09083238号</a><br>-->
</div>
</div>
</footer>
<!-- 代码高亮 -->
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shCore.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shLegacy.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shAutoloader.js"></script>
<link type="text/css" rel="stylesheet" href="/static/syntaxhighlighter/styles/shCoreDefault.css"/>
<script type="text/javascript" src="/static/syntaxhighlighter/src/my_start_1.js"></script>
</body>
</html>