17、Kubernetes - 实战：k8s之Calico网络策略
一、网络策略
网络策略（Network Policy ）是 Kubernetes 的一种资源。Network Policy 通过 Label 选择 Pod，并指定其他 Pod 或外界如何与这些 Pod 通信。
Pod的网络流量包含流入（Ingress）和流出（Egress）两种方向。默认情况下，所有 Pod 是非隔离的，即任何来源的网络流量都能够访问 Pod，没有任何限制。当为 Pod 定义了 Network Policy，只有 Policy 允许的流量才能访问 Pod。
Kubernetes的网络策略功能也是由第三方的网络插件实现的，因此，只有支持网络策略功能的网络插件才能进行配置网络策略，比如Calico、Canal、kube-router等等。
二、配置网络策略
在Kubernetes系统中，报文的流入和流出的核心组件是Pod资源，它们也是网络策略功能的主要应用对象。NetworkPolicy对象通过podSelector选择 一组Pod资源作为控制对象。NetworkPolicy是定义在一组Pod资源之上用于管理入站流量，或出站流量的一组规则，有可以是出入站规则一起生效，规则的生效模式通常由spec.policyTypes进行 定义。如下图：
默认情况下，Pod对象的流量控制是为空的，报文可以自由出入。在附加网络策略之后，Pod对象会因为NetworkPolicy而被隔离，一旦名称空间中有任何NetworkPolicy对象匹配了某特定的Pod对象，则该Pod将拒绝NetworkPolicy规则中不允许的所有连接请求，但是那些未被匹配到的Pod对象依旧可以接受所有流量。
就特定的Pod集合来说，入站和出站流量默认是放行状态，除非有规则可以进行匹配。还有一点需要注意的是，在spec.policyTypes中指定了生效的规则类型，但是在networkpolicy.spec字段中嵌套定义了没有任何规则的Ingress或Egress时，则表示拒绝入站或出站的一切流量。定义网络策略的基本格式如下：
apiVersion: networking.k8s.io/v1 #定义API版本
kind: NetworkPolicy 定义资源类型
metadata:
name: allow-myapp-ingress 定义NetwokPolicy的名字
namespace: default
spec: NetworkPolicy规则定义
podSelector: 匹配拥有标签app:myapp的Pod资源
matchLabels:
app: myapp
policyTypes ["Ingress"] NetworkPolicy类型，可以是Ingress，Egress，或者两者共存
ingress: 定义入站规则
- from:
- ipBlock: 定义可以访问的网段
cidr: 10.244.0.0/16
except: 排除的网段
- 10.244.3.0/24
- podSelector: 选定当前default名称空间，标签为app:myapp可以入站
matchLabels:
app: myapp
ports: 开放的协议和端口定义
- protocol: TCP
port: 80
该网络策略就是将default名称空间中拥有标签"app=myapp"的Pod资源开放80/TCP端口给10.244.0.0/16网段，并排除10.244.3.0/24网段的访问，并且也开放给标签为app=myapp的所有Pod资源进行访问。
三、实验案例
环境准备
创建linux和python两个ns并分别在这两个ns中创建pod资源来用于测试网络策略
root@k8s-deploy-harbor:~/manifest/network-policy# kubectl create ns linux
namespace/linux created
root@k8s-deploy-harbor:~/manifest/network-policy# kubectl create ns python
namespace/python created
root@k8s-deploy-harbor:~/manifest/network-policy
创建linux namespace中的资源
nginx-linux-ns.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: linux-nginx-deployment-label
name: linux-nginx-deployment
namespace: linux
spec:
replicas: 1
selector:
matchLabels:
app: linux-nginx-selector
template:
metadata:
labels:
app: linux-nginx-selector
spec:
containers:
- name: linux-nginx-container
image: harbor.zhrx.com/baseimages/nginx:alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
protocol: TCP
name: http
- containerPort: 443
protocol: TCP
name: https
---
kind: Service
apiVersion: v1
metadata:
labels:
app: linux-nginx-service-label
name: linux-nginx-service
namespace: linux
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
nodePort: 30004
- name: https
port: 443
protocol: TCP
targetPort: 443
nodePort: 30443
selector:
app: linux-nginx-selector
tomcat-linux-ns.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: linux-tomcat-app1-deployment-label
name: linux-tomcat-app1-deployment
namespace: linux
spec:
replicas: 1
selector:
matchLabels:
app: linux-tomcat-app1-selector
template:
metadata:
labels:
app: linux-tomcat-app1-selector
spec:
containers:
- name: linux-tomcat-app1-container
image: harbor.zhrx.com/baseimages/tomcat:7.0.94-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
protocol: TCP
name: http
---
kind: Service
apiVersion: v1
metadata:
labels:
app: linux-tomcat-app1-service-label
name: linux-tomcat-app1-service
namespace: linux
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
nodePort: 30005
selector:
app: linux-tomcat-app1-selector
创建资源
root@k8s-deploy-harbor:~/manifest/network-policy/linux-ns# kubectl apply -f nginx-linux-ns.yaml
deployment.apps/linux-nginx-deployment created
service/linux-nginx-service created
root@k8s-deploy-harbor:~/manifest/network-policy/linux-ns# kubectl apply -f tomcat-linux-ns.yaml
deployment.apps/linux-tomcat-app1-deployment created
service/linux-tomcat-app1-service created
root@k8s-deploy-harbor:~/manifest/network-policy/linux-ns# kubectl get pod,svc -n linux
NAME READY STATUS RESTARTS AGE
pod/linux-nginx-deployment-74fc689864-f78nb 1/1 Running 0 4m3s
pod/linux-tomcat-app1-deployment-7cc6bdd987-27wf5 1/1 Running 0 14s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/linux-nginx-service NodePort 10.68.187.8 <none> 80:30004/TCP,443:30443/TCP 4m3s
service/linux-tomcat-app1-service NodePort 10.68.170.248 <none> 80:30005/TCP 14s
root@k8s-master:~# kubectl exec -it linux-tomcat-app1-deployment-7cc6bdd987-27wf5 -n linux sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/usr/local/tomcat mkdir webapps/linux
/usr/local/tomcat echo "linux tomcat" > webapps/linux/index.jsp
创建python namespace中的资源
nginx-python-ns.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: python-nginx-deployment-label
name: python-nginx-deployment
namespace: python
spec:
replicas: 1
selector:
matchLabels:
app: python-nginx-selector
template:
metadata:
labels:
app: python-nginx-selector
project: python
spec:
containers:
- name: python-nginx-container
image: harbor.zhrx.com/baseimages/nginx:alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
protocol: TCP
name: http
- containerPort: 443
protocol: TCP
name: https
---
kind: Service
apiVersion: v1
metadata:
labels:
app: python-nginx-service-label
name: python-nginx-service
namespace: python
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
nodePort: 30014
- name: https
port: 443
protocol: TCP
targetPort: 443
nodePort: 30453
selector:
app: python-nginx-selector
project: python一个或多个selector，至少能匹配目标pod的一个标签
tomcat-python-ns.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: python-tomcat-app1-deployment-label
name: python-tomcat-app1-deployment
namespace: python
spec:
replicas: 1
selector:
matchLabels:
app: python-tomcat-app1-selector
template:
metadata:
labels:
app: python-tomcat-app1-selector
spec:
containers:
- name: python-tomcat-app1-container
image: harbor.zhrx.com/baseimages/tomcat:7.0.94-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
protocol: TCP
name: http
---
kind: Service
apiVersion: v1
metadata:
labels:
app: python-tomcat-app1-service-label
name: python-tomcat-app1-service
namespace: python
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
nodePort: 30015
selector:
app: python-tomcat-app1-selector
创建资源
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl apply -f nginx-python-ns.yaml
deployment.apps/python-nginx-deployment created
service/python-nginx-service created
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl apply -f tomcat-python-ns.yaml
deployment.apps/python-tomcat-app1-deployment created
service/python-tomcat-app1-service created
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl get pod,svc -n python
NAME READY STATUS RESTARTS AGE
pod/python-nginx-deployment-889b9fd95-6c9pv 1/1 Running 0 11s
pod/python-tomcat-app1-deployment-85d7bcb88c-xcrz4 1/1 Running 0 7s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/python-nginx-service NodePort 10.68.192.11 <none> 80:30014/TCP,443:30453/TCP 10s
service/python-tomcat-app1-service NodePort 10.68.88.145 <none> 80:30015/TCP 7s
root@k8s-master:~# kubectl exec -it python-tomcat-app1-deployment-85d7bcb88c-xcrz4 -n python sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/usr/local/tomcat mkdir webapps/python
/usr/local/tomcat echo "python tomcat" > webapps/python/index.jsp
在default namespace中创建测试pod
testpod-default.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: testpod-busybox-deployment-label
name: testpod-busybox-deployment
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: python-nginx-selector
project: python
template:
metadata:
labels:
app: python-nginx-selector
project: python
spec:
containers:
- name: testpod-busybox-container
image: busybox:latest
command:
- sleep
- "50000000"
imagePullPolicy: IfNotPresentc
创建资源
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl apply -f testpod-busybox.yaml
deployment.apps/testpod-busybox-deployment created
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl get pod
NAME READY STATUS RESTARTS AGE
testpod-busybox-deployment-8f4f66686-wvw6c 1/1 Running 0 49s
case1-ingress-podSelector
仅允许python namespace下拥有project: "python"标签的pod访问 拥有app: python-tomcat-app1-selector 标签的pod
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:
matchLabels:
app: python-tomcat-app1-selector对匹配到的目的Pod应用以下规则
ingress:入栈规则，如果指定目标端口就是匹配全部端口和协议，协议TCP, UDP, or SCTP
- from:
- podSelector:
matchLabels:
app: python-nginx-selector如果存在多个matchLabel条件，是or的关系，即要同时满足条件A、条件B、条件X
project: "python"
创建规则并验证
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl apply -f case1-ingress-podSelector.yaml
networkpolicy.networking.k8s.io/tomcat-access--networkpolicy created
#在python ns下的nginx pod中可以访问到tomcat
root@k8s-master:~# kubectl get svc -n python
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
python-nginx-service NodePort 10.68.192.11 <none> 80:30014/TCP,443:30453/TCP 39m
python-tomcat-app1-service NodePort 10.68.88.145 <none> 80:30015/TCP 39m
root@k8s-master:~# kubectl exec -it python-nginx-deployment-889b9fd95-6c9pv -n python sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ curl python-tomcat-app1-service/python/
python tomcat
#在linux ns下的nginx pod中不可访问
root@k8s-master:~# kubectl exec -it linux-nginx-deployment-74fc689864-f78nb -n linux sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ curl python-tomcat-app1-service.python.svc.cluster.local/python/
/ 阻塞无返回
case2-ingress-podSelector-ns-SinglePort
仅允许python namespace下拥有project: "python"标签的pod访问 拥有app: python-tomcat-app1-selector 标签的pod 的8080端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:
matchLabels:
app: python-tomcat-app1-selector
ingress:
- from:
- podSelector:
matchLabels:
app: python-nginx-selector指定访问源的匹配条件,如果存在多个matchLabel条件,是or的关系,即要同时满足条件A、条件B、条件X
project: "python"
ports:入栈规则，如果指定目标端口就是匹配全部端口和协议,协议TCP, UDP, or SCTP
- protocol: TCP
port: 8080允许通过TCP协议访问目标pod的8080端口,但是其它没有允许的端口将全部禁止访问
port: 80
创建规则并验证
root@k8s-deploy-harbor:~/manifest/network-policy/python-ns# kubectl apply -f case2-ingress-podSelector-ns-SinglePort.yaml
networkpolicy.networking.k8s.io/tomcat-access--networkpolicy created
# 通ns下nginx pod可以访问tomcat pod的8080端口
root@k8s-master:~# kubectl exec -it python-nginx-deployment-889b9fd95-6c9pv -n python sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ curl 172.20.36.69:8080/python/
python tomcat
case3-ingress-podSelector-ns-MultiPort
仅允许python namespace下拥有project: "python"标签的pod访问 拥有app: python-tomcat-app1-selector 标签的pod 的8080和8009端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:目标pod
matchLabels:
app: python-tomcat-app1-selector
ingress:
- from:
- podSelector:匹配源pod,matchLabels: {}为不限制源pod即允许所有pod,写法等同于resources(不加就是不限制)
matchLabels: {}
ports:入栈规则，如果指定目标端口就是匹配全部端口和协议，协议TCP, UDP, or SCTP
- protocol: TCP
port: 8080允许通过TCP协议访问目标pod的8080端口，但是其它没有允许的端口将全部禁止访问
- protocol: TCP
port: 8009
case4-ingress-podSelector-ns
允许当前namespace下所有pod的通信
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:目标pod
matchLabels: {}匹配所有目标pod
ingress:
- from:
- podSelector:匹配源pod,matchLabels: {}为不限制源pod即允许所有pod,写法等同于resources(不加就是不限制)
matchLabels: {}
case5-ingress-ipBlock
允许指定的ip网段访问tomcat pod的808端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:目标pod
matchLabels:
app: python-tomcat-app1-selector
ingress:
- from:
- ipBlock:
cidr: 10.200.0.0/16白名单，允许访问的地址范围，没有允许的将禁止访问目标pod
except:
- 10.200.218.0/24在以上范围内禁止访问的源IP地址
- 10.200.229.0/24在以上范围内禁止访问的源IP地址
- 10.200.230.239/32在以上范围内禁止访问的源IP地址
ports:入栈规则，如果指定目标端口就是匹配全部端口和协议，协议TCP, UDP, or SCTP
- protocol: TCP
port: 8080允许通过TCP协议访问目标pod的8080端口，但是其它没有允许的端口将全部禁止访问
case6-ingress-namespaceSelector
允许指定的ns访问当前ns下的所有pod 的某些端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomcat-access--networkpolicy
namespace: python
spec:
policyTypes:
- Ingress
podSelector:目标pod
matchLabels: {}允许访问python namespace 中的所有pod
# app: python-tomcat-app1-selector可以只允许访问python namespace中指定的pod
ingress:
- from:
- namespaceSelector:
matchLabels:
nsname: linux只允许指定的namespace访问
- namespaceSelector:
matchLabels:
nsname: python只允许指定的namespace访问
ports:入栈规则，如果指定目标端口就是匹配全部端口和协议，协议TCP, UDP, or SCTP
- protocol: TCP
port: 8080允许通过TCP协议访问目标pod的8080端口，但是其它没有允许的端口将全部禁止访问
port: 80
- protocol: TCP
port: 3306
- protocol: TCP
port: 6379
case7-Egress-ipBlock
允许指定pod访问目的ip网段的指定端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-access-networkpolicy
namespace: python
spec:
policyTypes:
- Egress
podSelector:目标pod选择器
matchLabels: 基于label匹配目标pod
app: python-tomcat-app1-selector匹配python namespace中app的值为python-tomcat-app1-selector的pod,然后基于egress中的指定网络策略进行出口方向的网络限制
egress:
- to:
- ipBlock:
cidr: 10.200.0.0/16允许匹配到的pod出口访问的目的CIDR地址范围
- ipBlock:
cidr: 172.31.7.106/32允许匹配到的pod出口访问的目的主机
ports:
- protocol: TCP
port: 80允许匹配到的pod访问目的端口为80的访问
- protocol: TCP
port: 53允许匹配到的pod访问目的端口为53 即DNS的解析
- protocol: UDP
port: 53允许匹配到的pod访问目的端口为53 即DNS的解析
case8-Egress-PodSelector
允许拥有app: python-nginx-selector标签的pod 访问 拥有app: python-tomcat-app1-selector 的pod的指定端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-access-networkpolicy
namespace: python
spec:
policyTypes:
- Egress
podSelector:目标pod选择器
matchLabels: 基于label匹配目标pod
app: python-nginx-selector匹配python namespace中app的值为python-tomcat-app1-selector的pod,然后基于egress中的指定网络策略进行出口方向的网络限制
egress:
- to:
- podSelector:匹配pod,matchLabels: {}为不限制源pod即允许所有pod,写法等同于resources(不加就是不限制)
matchLabels:
app: python-tomcat-app1-selector
ports:
- protocol: TCP
port: 8080允许80端口的访问
- protocol: TCP
port: 53允许DNS的解析
- protocol: UDP
port: 53
case9-Egress-namespaceSelector
允许拥有app: python-nginx-selector的pod 去访问指定ns下的pod 的指定端口
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-access-networkpolicy
namespace: python
spec:
policyTypes:
- Egress
podSelector:目标pod选择器
matchLabels: 基于label匹配目标pod
app: python-nginx-selector匹配python namespace中app的值为python-tomcat-app1-selector的pod,然后基于egress中的指定网络策略进行出口方向的网络限制
egress:
- to:
- namespaceSelector:
matchLabels:
nsname: python指定允许访问的目的namespace
- namespaceSelector:
matchLabels:
nsname: linux指定允许访问的目的namespace
ports:
- protocol: TCP
port: 8080允许80端口的访问
- protocol: TCP
port: 53允许DNS的解析
- protocol: UDP
port: 53