[TOC] 本文介绍calico网络插件如何对Pod限制网络带宽。 ### **更改Calico的配置** 更改calico的配置 ``` $ kubectl edit cm calico-config -n kube-system ``` 在plugins中添加如下内容： ``` { "type": "bandwidth", "capabilities": {"bandwidth": true} } ``` 如下是calico-config这个ConfigMap一个完整的示例： ``` apiVersion: v1 kind: ConfigMap metadata: name: calico-config namespace: kube-system data: calico_backend: bird typha_service_name: none veth_mtu: "1480" cni_network_config: | { "name": "k8s-pod-network", "cniVersion": "0.3.1", "plugins": [ { "type": "calico", "log_level": "info", "datastore_type": "kubernetes", "nodename": "__KUBERNETES_NODE_NAME__", "mtu": __CNI_MTU__, "ipam": { "type": "calico-ipam", "assign_ipv4": "true" }, "policy": { "type": "k8s" }, "kubernetes": { "kubeconfig": "__KUBECONFIG_FILEPATH__" } }, { "type": "portmap", "snat": true, "capabilities": {"portMappings": true} }, { "type": "bandwidth", "capabilities": {"bandwidth": true} } ] } ``` 然后重启一下calico-node ``` $ kubectl delete pod -n kube-system -l k8s-app=calico-node ``` 然后查看K8S节点上的`/etc/cni/net.d/10-calico.conflist`文件，应该已经更改过来 ### **创建Pod** 创建Pod，Pod的annotation中要添加如下两行内容： ``` apiVersion: v1 kind: Pod metadata: name: nginx annotations: kubernetes.io/ingress-bandwidth: 20M kubernetes.io/egress-bandwidth: 20M spec: containers: - name: nginx image: nginx:1.17.10 ``` 等待Pod处于运行状态 ``` $ kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx 1/1 Running 0 3s 172.26.192.16 10.50.208.48 <none> <none> ``` 然后在Pod所在的节点上，执行以下命令，查看是否有qdisc规则 ``` $ route -n | grep 172.26.192.16 172.26.192.16 0.0.0.0 255.255.255.255 UH 0 0 0 calic440f455693 $ tc qdisc show qdisc tbf 1: dev calic440f455693 root refcnt 2 rate 20Mbit burst 256Mb lat 25.0ms qdisc ingress ffff: dev calic440f455693 parent ffff:fff1 ---------------- qdisc tbf 1: dev 79c7 root refcnt 2 rate 20Mbit burst 256Mb lat 25.0ms ``` ### **测试** 下载iperf2测试工具，然后重命名为iperf，添加可执行权限 ``` $ wget https://iperf.fr/download/ubuntu/iperf_2.0.9 $ mv iperf_2.0.9 iperf $ chmod a+r+x ./iperf ``` 然后进行测试，速率应该会慢慢地限制在20M以内（注意，如果上面的容器没有使用nginx镜像，iperf命令可能会报错） ``` $ ./iperf -c 172.26.192.16 -p 80 -i 1 ------------------------------------------------------------ Client connecting to 172.26.192.16, TCP port 80 TCP window size: 12.0 MByte (default) ------------------------------------------------------------ [ 4] local 10.50.208.233 port 58697 connected with 172.26.192.16 port 80 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 258 MBytes 2.16 Gbits/sec [ 4] 1.0- 2.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 2.0- 3.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 3.0- 4.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 4.0- 5.0 sec 2.33 MBytes 19.5 Mbits/sec [ 4] 5.0- 6.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 6.0- 7.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 7.0- 8.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 8.0- 9.0 sec 2.27 MBytes 19.0 Mbits/sec [ 4] 9.0-10.0 sec 2.33 MBytes 19.5 Mbits/sec [ 4] 0.0-10.1 sec 278 MBytes 232 Mbits/sec ``` ### **参考** * https://docs.projectcalico.org/reference/cni-plugin/configuration#cni-network-configuration-lists * https://cloud.tencent.com/document/product/457/48766