缩容
- 1.k8s集群的缩容的流程
- 1.1 驱逐已经调度到该节点的Pod;
- 1.2 将驱逐节点的kubelet进行下线;
- 1.3 将驱逐节点的数据进行备份，迁移，重置之后在重新安装操作系统(避免数据泄露，将多余的磁盘进行格式化);
- 1.4 在master节点移除已经驱逐的节点- 2.驱逐节点实战案例
2.1 驱逐已经调度到该节点的Pod;
[root@master231 scheduler]# kubectl drain worker233 --ignore-daemonsets
node/worker233 cordoned
WARNING: ignoring DaemonSet-managed Pods: calico-system/calico-node-d4554, calico-system/csi-node-driver-8vj74, kube-system/kube-proxy-mbdf6, metallb-system/speaker-cpt7s
evicting pod default/scheduler-resources-6d6785785-wz9xq
evicting pod default/scheduler-resources-6d6785785-hmghm
evicting pod calico-system/calico-typha-595f8c6fcb-n7ffv
evicting pod default/scheduler-resources-6d6785785-l5nns
evicting pod default/scheduler-resources-6d6785785-vrch5
pod/calico-typha-595f8c6fcb-n7ffv evicted
pod/scheduler-resources-6d6785785-l5nns evicted
pod/scheduler-resources-6d6785785-vrch5 evicted
pod/scheduler-resources-6d6785785-hmghm evicted
pod/scheduler-resources-6d6785785-wz9xq evicted
node/worker233 drained
[root@master231 scheduler]#
[root@master231 scheduler]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master231 Ready control-plane,master 3d1h v1.23.17
worker232 Ready <none> 3d v1.23.17
worker233 Ready,SchedulingDisabled <none> 3d v1.23.17
[root@master231 scheduler]#
- 2.2 将驱逐节点的kubelet进行下线;
[root@worker233 ~]# systemctl disable --now kubelet.service
Removed /etc/systemd/system/multi-user.target.wants/kubelet.service.
[root@worker233 ~]#
- 2.3 将驱逐节点的数据进行备份，迁移，重置之后在重新安装操作系统(避免数据泄露，将多余的磁盘进行格式化);
[root@worker233 ~]# kubeadm reset -f
[preflight] Running pre-flight checks
W0410 12:03:58.005183 211122 removeetcdmember.go:80] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d
The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.
If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.
The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
[root@worker233 ~]#
- 2.4 在master节点移除已经驱逐的节点
[root@master231 scheduler]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master231 Ready control-plane,master 3d1h v1.23.17
worker232 Ready <none> 3d v1.23.17
worker233 NotReady,SchedulingDisabled <none> 3d v1.23.17
[root@master231 scheduler]#
[root@master231 scheduler]#
[root@master231 scheduler]# kubectl delete nodes worker233
node "worker233" deleted
[root@master231 scheduler]#
[root@master231 scheduler]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master231 Ready control-plane,master 3d1h v1.23.17
worker232 Ready <none> 3d v1.23.17
[root@master231 scheduler]# 扩容
- 1.扩容流程
- 1.1 被扩容节点需要安装docker|containerd，kubeadm，Kubectl，kubelet工具包，及基础优化(核心数大于2core，内核调优，禁用swap分区)等;
- 1.2 服务端需要创建token，将来被加入集群的worker基于该token进行认证;
- 1.3 将kubelet配置开机自启动;
- 1.4 使用kubeadm join加入集群（bootstrap阶段）;
- 1.5 管理节点查看验证;- 2.kubeadm管理token实战
2.1 创建token
[root@master231 scheduler]# kubeadm token create
7y9a6t.c4n40ljuec10tk6k
[root@master231 scheduler]#
2.2 查看token列表
[root@master231 scheduler]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
7y9a6t.c4n40ljuec10tk6k 23h 2025-04-11T06:44:44Z authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
[root@master231 scheduler]#
2.3 删除token
[root@master231 scheduler]# kubeadm token delete 7y9a6t
bootstrap token "7y9a6t" deleted
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token list
[root@master231 scheduler]#
2.4 临时生成token打印终端但不创建
[root@master231 scheduler]# kubeadm token generate
ux5xbj.tw0p1t82022577ko
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token generate
eaeebd.qnzwh0kb1ccuct8v
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token generate
p3k7bd.oo8brc14atcvjshs
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token list
[root@master231 scheduler]#
2.5 创建token可以自定义token
[root@master231 scheduler]# kubeadm token create oldboy.laxjason --print-join-command --ttl 0
kubeadm join 10.0.0.231:6443 --token oldboy.laxjason --discovery-token-ca-cert-hash sha256:2617b95e2ce0c94389031841fab9801e5724ed544cd9a60dcd285a9fa7a1b10b
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
oldboy.laxjason <forever> <never> authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
[root@master231 scheduler]# - 3.扩容k8s节点
3.1 被扩容节点需要安装docker|containerd，kubeadm，Kubectl，kubelet工具包，及基础优化(核心数大于2core，内核调优，禁用swap分区)等;
3.2 服务端需要创建token，将来被加入集群的worker基于该token进行认证;
[root@master231 scheduler]# kubeadm token create oldboy.laxjason --print-join-command --ttl 0
kubeadm join 10.0.0.231:6443 --token oldboy.laxjason --discovery-token-ca-cert-hash sha256:2617b95e2ce0c94389031841fab9801e5724ed544cd9a60dcd285a9fa7a1b10b
[root@master231 scheduler]#
[root@master231 scheduler]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
oldboy.laxjason <forever> <never> authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
[root@master231 scheduler]#
3.3 将kubelet配置开机自启动;
[root@worker233 ~]# systemctl is-enabled kubelet
disabled
[root@worker233 ~]#
[root@worker233 ~]# systemctl enable --now kubelet
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.
[root@worker233 ~]#
[root@worker233 ~]# systemctl is-enabled kubelet
enabled
[root@worker233 ~]#
3.4 使用kubeadm join加入集群（bootstrap阶段）;
[root@worker233 ~]# kubeadm join 10.0.0.231:6443 --token oldboy.laxjason --discovery-token-ca-cert-hash sha256:2617b95e2ce0c94389031841fab9801e5724ed544cd9a60dcd285a9fa7a1b10b
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0410 14:50:44.610565 213085 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@worker233 ~]#
3.5 管理节点查看验证
[root@master231 scheduler]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master231 Ready control-plane,master 3d3h v1.23.17
worker232 Ready <none> 3d3h v1.23.17
worker233 Ready <none> 30s v1.23.17
[root@master231 scheduler]#
[root@master231 scheduler]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master231 Ready control-plane,master 3d3h v1.23.17 10.0.0.231 <none> Ubuntu 22.04.4 LTS 5.15.0-136-generic docker://20.10.24
worker232 Ready <none> 3d3h v1.23.17 10.0.0.232 <none> Ubuntu 22.04.4 LTS 5.15.0-119-generic docker://20.10.24
worker233 Ready <none> 31s v1.23.17 10.0.0.233 <none> Ubuntu 22.04.4 LTS 5.15.0-119-generic docker://20.10.24
[root@master231 scheduler]# 周期性备份etcd数据
- 1.思路
- 1.备份工具etcdctl
- 2.使用cj控制器
- 3.备份时需要使用证书- 2.编写Dockerfile
[root@master231 backup]# cp /etc/kubernetes/pki/etcd/{ca.crt,peer.crt,peer.key} /usr/local/bin/etcdctl ./
[root@master231 backup]#
[root@master231 backup]# ll
total 18016
drwxr-xr-x 2 root root 4096 Apr 10 15:16 ./
drwxr-xr-x 4 root root 4096 Apr 10 15:05 ../
-rw-r--r-- 1 root root 1086 Apr 10 15:09 ca.crt
-rw-r--r-- 1 root root 219 Apr 10 15:16 Dockerfile
-rwxr-xr-x 1 root root 18419864 Apr 10 15:15 etcdctl*
-rw-r--r-- 1 root root 1200 Apr 10 15:09 peer.crt
-rw------- 1 root root 1675 Apr 10 15:09 peer.key
[root@master231 backup]#
[root@master231 backup]# cat Dockerfile
FROM harbor250.violet.com/violet-xiuxian/apps:v1
MAINTAINER JasonYin
LABEL school=violet \
class=linux96
COPY etcdctl /usr/local/bin/
COPY ca.crt peer.crt peer.key /
# CMD ["tail","-f","/etc/hosts"]
CMD ["/bin/sh","-c","etcdctl --endpoints=10.0.0.231:2379 --cacert=/ca.crt --cert=/peer.crt --key=/peer.key snapshot save /tmp/violet-etcd-`date +%F-%T`.backup"]
[root@master231 backup]#
- 3.测试验证
[root@master231 backup]# docker build -t etcd-backup:v0.1 .
Sending build context to Docker daemon 18.43MB
Step 1/6 : FROM harbor250.violet.com/violet-xiuxian/apps:v1
# Executing 1 build trigger
---> Using cache
---> 3702ed09067c
Step 2/6 : MAINTAINER JasonYin
---> Using cache
---> e45c3391c338
Step 3/6 : LABEL school=violet class=linux96
---> Using cache
---> d713219b897c
Step 4/6 : COPY etcdctl /usr/local/bin/
---> Using cache
---> f7bdd47d599d
Step 5/6 : COPY ca.crt peer.crt peer.key /
---> Using cache
---> 0a998727add5
Step 6/6 : CMD ["/bin/sh","-c","etcdctl --endpoints=10.0.0.231:2379 --cacert=/ca.crt --cert=/peer.crt --key=/peer.key snapshot save /tmp/violet-etcd-`date +%F-%T`.backup"]
---> Using cache
---> fab91124fe23
Successfully built fab91124fe23
Successfully tagged etcd-backup:v0.1
[root@master231 backup]#
[root@master231 backup]# docker run -d --name etcd-bak -v /xixi:/tmp etcd-backup:v0.1
6e60db3b0d27620655343d91d24300e82b10bc43c8aebf07a6debc4fa5e63cfa
[root@master231 backup]#
[root@master231 backup]#
[root@master231 backup]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6e60db3b0d27 etcd-backup:v0.1 "/docker-entrypoint.…" 17 seconds ago Exited (0) 16 seconds ago etcd-bak
[root@master231 backup]#
[root@master231 backup]#
[root@master231 backup]# ll /xixi/
total 18288
drwxr-xr-x 2 root root 4096 Apr 10 15:24 ./
drwxr-xr-x 22 root root 4096 Apr 10 15:22 ../
-rw------- 1 root root 9355296 Apr 10 15:24 violet-etcd-2025-04-10-07:24:21.backup
[root@master231 backup]# 图形化管理K8S集群kuboard实战
官网地址:
https://kuboard.cn/
- 1.部署kuboard
[root@master231 kuboard]# wget https://addons.kuboard.cn/kuboard/kuboard-v3-swr.yaml
[root@master231 kuboard]# kubectl apply -f kuboard-v3-swr.yaml
namespace/kuboard created
configmap/kuboard-v3-config created
serviceaccount/kuboard-boostrap created
clusterrolebinding.rbac.authorization.k8s.io/kuboard-boostrap-crb created
daemonset.apps/kuboard-etcd created
deployment.apps/kuboard-v3 created
service/kuboard-v3 created
[root@master231 kuboard]#
[root@master231 kuboard]# kubectl get pods -n kuboard -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kuboard-agent-2-55b9bfbb7c-89nff 1/1 Running 2 (2m43s ago) 2m55s 10.100.0.56 master231 <none> <none>
kuboard-agent-6f4885bcd7-7xzz8 1/1 Running 2 (2m44s ago) 2m55s 10.100.0.55 master231 <none> <none>
kuboard-etcd-trt8q 1/1 Running 0 3m58s 10.0.0.231 master231 <none> <none>
kuboard-v3-685dc9c7b8-bhqfw 1/1 Running 0 3m58s 10.100.0.54 master231 <none> <none>
[root@master231 kuboard]#
镜像地址:
http://192.168.16.253/Resources/Kubernetes/Project/kuboard/kuboard-on-k8s/- 2.访问 Kuboard
[root@master231 kuboard]# kubectl get svc -n kuboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kuboard-v3 NodePort 10.200.117.52 <none> 80:30080/TCP,10081:30081/TCP,10081:30081/UDP 4m36s
[root@master231 kuboard]#
在浏览器中打开链接 http://10.0.0.233:30080
输入初始用户名和密码，并登录
用户名： admin
密码： Kuboard123kuboard如果k8s部署不成功，可以考虑使用docker部署
https://kuboard.cn/install/install-k8s.html#%E5%AE%89%E8%A3%85-kuboard-spray
docker run -d \
–privileged \
–restart=unless-stopped \
–name=kuboard-spray \
-p 80:80/tcp \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/kuboard-spray-data:/data \
swr.cn-east-2.myhuaweicloud.com/kuboard/kuboard-spray:latest-amd64
