Use NetworkPolicy in CCE Cluster

Last Updated：2021-04-27

NetworkPolicy is a resource provided by K8S, which is used to define the Pod-based network isolation policy. It describes whether a group of Pods can communicate with other groups of Pods and other Endpoints. This article mainly demonstrates how to perform the NetworkPolicy function on CCE through the open-source tool kube-router.

kube-router

kube-router is a container network solution of kubernetes, whose official website and code address are shown as follows:

Official Website:https://www.kube-router.io
Project:https://github.com/cloudnativelabs/kube-router

kube-router can perform three functions:

Pod Networking;
IPVS/LVS based service proxy;
Network Policy Controller.

CCE has a special container network implementation scheme. This article mainly introduces the kube-router’s Network Policy Controller feature.

Deploy kube-router

Deploy kube-router and YAML on the CCE K8S cluster as follows:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-router-cfg
  namespace: kube-system
  labels:
    tier: node
    k8s-app: kube-router
data:
  cni-conf.json: |
    {
      "name":"kubernetes",
      "type":"bridge",
      "bridge":"kube-bridge",
      "isDefaultGateway":true,
      "ipam": {
        "type":"host-local"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-router
  namespace: kube-system
  labels:
    k8s-app: kube-router
spec:
  selector:
    matchLabels:
      k8s-app: kube-router
  template:
    metadata:
      labels:
        k8s-app: kube-router
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      containers:
      - name: kube-router
        image: docker.io/cloudnativelabs/kube-router
        args: ["--run-router=false", "--run-firewall=true", "--run-service-proxy=false", "--kubeconfig=/root/.kube/config"]
        securityContext:
          privileged: true
        imagePullPolicy: Always
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        livenessProbe:
          httpGet:
            path: /healthz
            port: 20244
          initialDelaySeconds: 10
          periodSeconds: 3
        volumeMounts:
        - name: lib-modules
          mountPath: /lib/modules
          readOnly: true
        - name: cni-conf-dir
          mountPath: /etc/cni/net.d
        - name: kubeconfig
          mountPath: /root/.kube/config
          readOnly: true
      initContainers:
      - name: install-cni
        image: busybox
        imagePullPolicy: Always
        command:
        - /bin/sh
        - -c
        - set -e -x;
          if [ ! -f /etc/cni/net.d/10-kuberouter.conf ]; then
            TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
            cp /etc/kube-router/cni-conf.json ${TMP};
            mv ${TMP} /etc/cni/net.d/10-kuberouter.conf;
          fi
        volumeMounts:
        - name: cni-conf-dir
          mountPath: /etc/cni/net.d
        - name: kube-router-cfg
          mountPath: /etc/kube-router
      hostNetwork: true
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - effect: NoSchedule
        key: node.kubernetes.io/not-ready
        operator: Exists
      volumes:
      - name: lib-modules
        hostPath:
          path: /lib/modules
      - name: cni-conf-dir
        hostPath:
          path: /etc/cni/net.d
      - name: kube-router-cfg
        configMap:
          name: kube-router-cfg
      - name: kubeconfig
        hostPath:
          path: /root/.kube/config

Example Description

1. Create namespaces

$kubectl create namespace production
$kubectl create namespace staging

2. Start nginx service

Create the nginx deployment in different namespaces.

$kubectl apply -f nginx.yaml --namespace=production
$kubectl apply -f nginx.yaml --namespace=staging

The YAML of nginx.yaml is shown as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: hub.baidubce.com/cce/nginx-alpine-go:latest
        ports:
        - containerPort: 80

Verify that the Pod is started successfully:

# staging 环境
$kubectl get pods -n staging
NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-7fbd5f4c55-2xgd4   1/1       Running   0          45s
nginx-deployment-7fbd5f4c55-5xr75   1/1       Running   0          45s
nginx-deployment-7fbd5f4c55-fn6lr   1/1       Running   0          20m

# productionn 环境
$kubectl get pods -n production
NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-7fbd5f4c55-m764f   1/1       Running   0          10s
nginx-deployment-7fbd5f4c55-pdhhz   1/1       Running   0          10s
nginx-deployment-7fbd5f4c55-r98w5   1/1       Running   0          20m

When NetworkPolicy is not set, all Pods can access each other and ping PodIP directly.

Network Policy Testing

1. Default deny all ingress traffic

Prohibit the Pod in namespace=staging from being accessed.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: staging
spec:
  podSelector: {}
  policyTypes:
  - Ingress

The meaning of fields is as follows:

PodSelector: Select the Pod to be isolated.
policyTypes: The policy type. The NetworkPolicy classifies the traffic into ingress and egress traffic, i.e., ingress direction and egress direction. If it is not specified, indicate it is not idle.
Ingress: The ingress direction and whitelist. You need to specify “from” and “ports”, i.e., source and destination port number. There are three types of “from”, i.e., ipBlock/namespaceSelector/podSelector；
egress: The egress direction and whitelist. Similar to ingress, you need to specify “to” and “ports” for egress, i.e., destination and destination port number.

After above NetworkPolicy is created successfully, you can access PodIP in the namespace=staging in any Pod. If you find that it fails to access, e.g., access it from Pod in the production:

$kubectl exec -it nginx-deployment-7fbd5f4c55-m764f /bin/sh -n production
/ # ping 172.16.0.92
PING 172.16.0.92 (172.16.0.92): 56 data bytes

2. Default allow all ingress traffic

Allow the Pod in namespace=staging to be accessed.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all
  namespace: staging
spec:
  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

3. Default deny all egress traffic

Prohibit the Pod in namespace=production from external access.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Egress

4. Default allow all egress traffic

Allow the Pod in namespace=production for external access.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all
  namespace: production
spec:
  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

5. Default deny all ingress and all egress traffic

Prohibit the ingress and egress traffic of all Pods:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Traffic Access

Create VPC-CNI Mode Cluster

百度智能云

Cloud Container Engine