用 Calico 网络策略设置主机 node 防火墙规则

Cloudpods 的服务运行在一个 Kubernetes 集群之上，该 Kubernets 集群的网络方案采用了Calico。因此运行 Cloudpods 服务的节点的 iptables 规则被 Calico 接管。这就导致我们在 Cloudpods 服务节点上配置的防火墙规则会被 Calico 配置的 iptables 规则覆盖，导致防火墙规则不生效。本文介绍如何使用 Calico 的 HostEndpoint 和 GlobalNetworkPolicy 来设置主机节点的防火墙规则。

1 、准备 calicoctl 工具

下载二进制

curl -O -L https://github.com/projectcalico/calicoctl/releases/download/v3.12.1/calicoctl
chmod +x calicoctl

设置环境变量

export DATASTORE_TYPE=kubernetes
export KUBECONFIG=/etc/kubernetes/admin.conf

2 、配置 HostEndpoint 规则

对每一台主机的每个需要控制防火墙规则接口，定义对应的 HostEndpoint 规则

- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: <node_name>-<interface_name>
    labels:
      role: master
      env: production
  spec:
    interfaceName: <interface_name>
    node: <node_name>
    expectedIPs: ["<interface_ip>"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: <node_name>-<interface_name>
    labels:
      role: master
      env: production
  spec:
    interfaceName: <interface_name>
    node: <node_name>
    expectedIPs: ["<interface_ip>"]

应用该规则：

./calicoctl apply -f hep.yaml

3 、定义网络规则

定义好 HostEndpoint 之后，采用 Calico 的 GlobalNetworkPolicy 定义防火墙规则。

- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: <whitelist_gnp_name>
  spec:
    order: 10
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Allow
        protocol: TCP
        source:
          nets: [<src_net_block1>, <src_net_block2>]
        destination:
          ports: [<dst_port1>, <dst_port2>]
    selector: "role==\"master\""
- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: drop-other-ingress
  spec:
    order: 20
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Deny
    selector: "role==\"master\""

应用规则

./calicoctl apply -f gnp.yaml

4. failSafe 机制

为防止用户错误配置导致 node 无法网络访问的风险，calico 设计了 failSafe 机制，即在用户编写规则有误的情况下，部分端口也不会被封禁，导致节点功能失效。这里是 FailSafe 端口的信息： https://docs.projectcalico.org/reference/host-endpoints/failsafe

5. 配置举例

举例：master 节点的外网端口只允许 80 和 443 端口，其他都禁止：

HostEndpoint 定义：

- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master1-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master1
    expectedIPs: ["120.133.60.219"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master2-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master2
    expectedIPs: ["120.133.60.220"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master3-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master3
    expectedIPs: ["120.133.60.221"]

GlobalNetworkPolicy 定义

- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: allow-http-https-traffic-only
  spec:
    order: 10
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Allow
        protocol: TCP
        destination:
          ports: [80,443]
    selector: "role==\"master\" && type==\"external\""
- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: drop-other-ingress
  spec:
    order: 20
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Deny

作者: 云联壹云小助手

GitHub: https://github.com/yunionio/cloudpods

开源地址： https://www.cloudpods.org/

Cloudpods 是一个开源的 Golang 实现的云原生的多云和混合云融合平台。Cloudpods 不仅可以管理本地的虚拟机和物理机资源，还可以管理其他公有云和私有云平台的资源。