kubeadm部署 · Kubernetes

[TOC] ## 修改主机名 ```shell sudo hostnamectl set-hostname <HOSTNAME> ``` >[info] `<HOSTNAME>` 根据实际的填写 ## 停止防火墙 ```shell sudo systemctl stop firewalld && sudo systemctl disable firewalld sudo setenforce 0 sudo sed -ri 's/^(SELINUX)=.*$/\1=disabled/' /etc/selinux/config ``` ## 关闭swap分区 ```shell sudo swapoff -a sudo sed -i '/ swap / s/^$.*$$/#\1/g' /etc/fstab ``` ## 设置内核模块 ```shell cat <<-EOF | sudo tee /etc/sysconfig/modules/ipvs.modules > /dev/null #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack modprobe -- br_netfilter EOF sudo chmod 755 /etc/sysconfig/modules/ipvs.modules && sudo bash /etc/sysconfig/modules/ipvs.modules ``` ## 设置的 sysctl 参数 ```shell cat <<-EOF | sudo tee /etc/sysctl.d/kubernetes.conf > /dev/null # 二层的网桥在转发包时也会被iptables的FORWARD规则所过滤 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 # kernel转发功能 net.ipv4.ip_forward = 1 # 允许将TIME-WAIT sockets重新用于新的TCP连接，默认为0，表示关闭 net.ipv4.tcp_tw_reuse = 1 # TCP连接中TIME-WAIT sockets的快速回收, 默认是0，表示关闭。对于位于NAT设备(容器转发)后面的Client来说，就是一场灾难 net.ipv4.tcp_tw_recycle = 0 # 允许系统打开的端口范围，即用于向外连接的端口范围 net.ipv4.ip_local_port_range = 32768 65535 # kernel中最多存在的TIME_WAIT数量, 默认是4096 net.ipv4.tcp_max_tw_buckets = 65535 # 控制系统是否开启对数据包源地址的校验(0 不校验) net.ipv4.conf.all.rp_filter = 0 # 开启ipv6路由转发 net.ipv6.conf.all.forwarding = 1 # 开启ipv4路由转发 net.ipv4.conf.all.forwarding = 1 # FIN-WAIT-2状态保持时间 net.ipv4.tcp_fin_timeout = 15 EOF sudo sysctl -p /etc/sysctl.d/kubernetes.conf ``` ## 安装containerd 请参考博客园 [containerd安装](https://www.cnblogs.com/jiaxzeng/p/16708491.html) 文章 ## 添加 k8s 源 ```shell cat <<-EOF | sudo tee /etc/yum.repos.d/kubernetes.repo > /dev/null [kubernetes] name=Aliyun-kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0 EOF sudo yum clean all && sudo yum makecache ``` ## 安装软件 ``` sudo yum install -y kubelet-1.23.3-0.x86_64 kubeadm-1.23.3-0.x86_64 kubectl-1.23.3-0.x86_64 ipvsadm conntrack-tools sudo systemctl enable kubelet.service ``` >[info] 查看 kubeadm 所有版本 `yum list kubeadm --showduplicates` ## 拉取k8s组件镜像 ```shell kubeadm config images pull --kubernetes-version=v1.23.3 --image-repository registry.aliyuncs.com/google_containers ``` ## 初始化集群配置 ```yaml --- apiVersion: kubeadm.k8s.io/v1beta3 kind: InitConfiguration nodeRegistration: criSocket: /run/containerd/containerd.sock --- apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration kubernetesVersion: 1.23.3 controlPlaneEndpoint: 172.139.20.100 imageRepository: registry.aliyuncs.com/google_containers networking: serviceSubnet: 10.96.0.0/12 podSubnet: 172.26.0.0/16 --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs ``` ## 初始化集群 ```shell sudo kubeadm init --config kubeadm-config.yaml --upload-certs --node-name 主机IP地址 ``` >[warning] **注意**：containerd 包含 cni 相关命令和配置可能影响安装。 > *确认方法*：查看 `/opt/cni/bin` 和 `/etc/cni/net.d` 是否有文件。确认时非空目录的话，需要删除后在执行创建集群 ## 安装calico插件 ### 下载manifest文件 ```shell mkdir /etc/kubernetes/addons curl https://projectcalico.docs.tigera.io/archive/v3.23/manifests/calico-etcd.yaml -o /etc/kubernetes/addons/calico.yaml ``` ### 生成calico证书 >[info] etcd证书在master节点上才有。 ```shell pki_dir=/etc/kubernetes/pki/etcd mkdir -p ${pki_dir} && cd ${pki_dir} openssl genrsa -out calico-etcd-client.key 2048 cat <<-EOF | sudo tee calico-etcd-client-csr.conf > /dev/null [ req ] default_bits = 2048 prompt = no default_md = sha256 distinguished_name = dn [ dn ] C = CN ST = Guangdong L = Guangzhou CN = calico-etcd-client [ v3_ext ] keyUsage=Digital Signature, Key Encipherment extendedKeyUsage=clientAuth basicConstraints=CA:FALSE authorityKeyIdentifier=keyid:always EOF openssl req -new -key calico-etcd-client.key -out calico-etcd-client.csr -config calico-etcd-client-csr.conf openssl x509 -req -in calico-etcd-client.csr -CA /etc/kubernetes/pki/etcd/ca.crt -CAkey /etc/kubernetes/pki/etcd/ca.key \ -CAcreateserial -out calico-etcd-client.crt -days 36500 \ -extensions v3_ext -extfile calico-etcd-client-csr.conf -sha256 openssl verify -CAfile /etc/kubernetes/pki/etcd/ca.crt ${pki_dir}/calico-etcd-client.crt [ $? -eq 0 ] && rm -rf ${pki_dir}/{calico-etcd-client-csr.conf,calico-etcd-client.csr} ``` ### 修改calico配置文件 1. 修改calico连接etcd的地址 >[info] 修改etcd地址 ```bash sed -ri 's@http://<ETCD_IP>:<ETCD_PORT>@https://192.168.32.127:2379,https://192.168.32.128:2379,https://192.168.32.129:2379@g' /etc/kubernetes/addons/calico.yaml ``` 2. 修改calico连接etcd证书 ```bash ETCD_CA=$(cat /etc/kubernetes/pki/etcd/ca.crt | base64 -w 0) ETCD_CERT=$(cat /etc/kubernetes/pki/etcd/calico-etcd-client.crt | base64 -w 0) ETCD_KEY=$(cat /etc/kubernetes/pki/etcd/calico-etcd-client.key | base64 -w 0) sed -ri "s@# etcd-ca: null@etcd-ca: ${ETCD_CA}@g" /etc/kubernetes/addons/calico.yaml sed -ri "s@# etcd-cert: null@etcd-cert: ${ETCD_CERT}@g" /etc/kubernetes/addons/calico.yaml sed -ri "s@# etcd-key: null@etcd-key: ${ETCD_KEY}@g" /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_ca: ""@etcd_ca: "/calico-secrets/etcd-ca"@g' /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_cert: ""@etcd_cert: "/calico-secrets/etcd-cert"@g' /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_key: ""@etcd_key: "/calico-secrets/etcd-key"@g' /etc/kubernetes/addons/calico.yaml ``` 3. 设置calico网段地址将 DaemonSet 类型，calico-node 的环境变量 `CALICO_IPV4POOL_CIDR` >[info] 默认是192.168.0.0/16地址 ```yaml - name: CALICO_IPV4POOL_CIDR value: "10.244.0.0/16" ``` ## 卸载集群 ```shell kubectl drain <node name> --delete-emptydir-data --force --ignore-daemonsets kubeadm reset -f rm -rf /etc/cni/net.d iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X ipvsadm -C kubectl delete node <node name> ```