kubernetes笔记-亲和性调度
2019-04-25 本文已影响121人
潘猛_9f76
以DaemonSet为例，目前有三种方法来指定想要调度的Node：
nodeSelector：只调度到匹配指定label的Node上
nodeAffinity：功能更丰富的Node选择器，支持集合操作和多个选择器
podAffinity：调度到满足条件的Pod所在的Node上
1、为Node添加label workload=true
# kubectl label node kube-node2 workload=true
# kubectl label node kube-node3 workload=true
# kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
kube-node1 Ready master 9d v1.14.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kube-node1,kubernetes.io/os=linux,node-role.kubernetes.io/master=
kube-node2 Ready 9d v1.14.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kube-node2,kubernetes.io/os=linux,workload=true
kube-node3 Ready 8d v1.14.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kube-node3,kubernetes.io/os=linux,workload=true
2、使用nodeSelector调度到包含标签workload='true'的Node上
注意true要加上引号
apiVersion:extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
nodeSelector:
workload: 'true'
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy: IfNotPresent
查看Daemonset的调度情况
# kubectl getpods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-dmknb 1/1 Running 0 49s 10.244.119.173 kube-node3
testpod-sjhl6 1/1 Running 0 49s 10.244.1.39 kube-node2
3、使用 nodeAffinity调度到包含标签workload:'true'的Node上
编写YAML文件：
apiVersion:extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values:
- 'true'
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy: IfNotPresent
查看Daemonset的调度情况
# kubectl getpods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-22mdb 1/1 Running 0 5s 10.244.119.174 kube-node3
testpod-v9xzn 1/1 Running 0 5s 10.244.1.40 kube-node2
修改DaemonSet，增加匹配条件kubernetes.io/os=linux
apiVersion:extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values:
- 'true'
- key: kubernetes.io/os
operator: In
values:
- linux
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy: IfNotPresent
查看Daemonset的调度情况
可以看到还是调度到kube-node3和kube-node2，说明两个匹配条件为AND关系
# kubectl get pods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-7cd6m 1/1 Running 0 11s 10.244.1.45 kube-node2
testpod-swpn8 1/1 Running 0 11s 10.244.119.180 kube-node3
修改yaml文件，增加matchFields匹配条件metadata.name=kube-node1
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values:
- 'true'
- key: kubernetes.io/os
operator: In
values:
- linux
-matchFields:
- key: metadata.name
operator: In
values:
- kube-node1
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy:IfNotPresent
查看Daemonset的调度情况
可以看到DaemonSet调度到了3个Node上，说明matchExpressions和matchFields之间是或的关系
# kubectl get pods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-42crx 1/1 Running 0 8s 10.244.0.12 kube-node1
testpod-6bqhp 1/1 Running 0 8s 10.244.119.181 kube-node3
testpod-h7296 1/1 Running 0 8s 10.244.1.46 kube-node2
同时指定了nodeSelector和nodeAffinity
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
nodeSelector:
workload: 'true'
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values:
- 'true'
- key: kubernetes.io/os
operator: In
values:
- linux
-matchFields:
- key: metadata.name
operator: In
values:
- kube-node1
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy:IfNotPresent
查看Daemonset的调度情况
# kubectl get pods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-bmxjt 1/1 Running 0 6m8s 10.244.119.182 kube-node3
testpod-cltnt 1/1 Running 0 6m8s 10.244.1.47 kube-node2
image.png
4、podAffinity调度
将Pod调度到拥有指定标签的Pod所在的Node及与该Node具有相同topologyKey标签value的Node上.查看当前pod的标签
# kubectl getpod --show-labels
NAME READY STATUS RESTARTS AGE LABELS
busybox-httpd-7cf56cb476-dxnmj 1/1 Running 1 21h pod-template-hash=7cf56cb476,run=busybox-httpd
busybox-httpd-7cf56cb476-ghgjr 1/1 Running 1 21h pod-template-hash=7cf56cb476,run=busybox-httpd
查看当前Pod所在Node
# kubectl getpod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox-httpd-7cf56cb476-dxnmj 1/1 Running 1 21h 10.244.1.25 kube-node2
busybox-httpd-7cf56cb476-ghgjr 1/1 Running 1 21h 10.244.119.161 kube-node3
编写YAML文件，通过podAffinity调度到kube-node2和kube-node3上。
apiVersion:extensions/v1beta1
kind: DaemonSet
metadata:
name: testpod
spec:
template:
metadata:
labels:
app: fortest
spec:
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: run
operator: In
values:
- busybox-httpd
topologyKey: kubernetes.io/hostname
containers:
- name: busybox-1
image: busybox:v1
command:
- /bin/sh
- -c
- 'sleep 365d'
imagePullPolicy: IfNotPresent
查看Daemonset的调度情况
# kubectl get pods -o wide -l app=fortest
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-2zrxj 1/1 Running 0 4m47s 10.244.119.175 kube-node3
testpod-8fzkc 0/1 Pending 0 4m47s <none> <none> <none> <none>
testpod-gwdsk 1/1 Running 0 4m47s 10.244.1.41 kube-node2
image.png
扩容deployment busybox-httpd，使得3个node都有busybox-httpd的pod，可以看到DasmonSet的3个Pod均变为running状态
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox-httpd-7cf56cb476-bzlcf 1/1 Running 0 14s 10.244.0.10 kube-node1
busybox-httpd-7cf56cb476-dxnmj 1/1 Running 1 22h 10.244.1.25 kube-node2
busybox-httpd-7cf56cb476-ghgjr 1/1 Running 1 22h 10.244.119.161 kube-node3
testpod-2zrxj 1/1 Running 0 7m7s 10.244.119.175 kube-node3
testpod-8fzkc 1/1 Running 0 7m7s 10.244.0.9 kube-node1
testpod-gwdsk 1/1 Running 0 7m7s 10.244.1.41 kube-node2
image.png
修改topologyKey为failure-domain.beta.kubernetes.io/zone，由于3个Node都没有这个标签，所以DaemonSet中没有Pod处于running状态
# kubectl get pods -l app=fortest
NAME READY STATUS RESTARTS AGE
testpod-7czf5 0/1 Pending 0 28m
testpod-hp6vc 0/1 Pending 0 28m
testpod-tqkrc 0/1 Pending 0 28m
image.png
逐步添加failure-domain.beta.kubernetes.io/zone到各个Node上，最终3个Pod都会进入Running状态
image.png
# kubectl label nodeskube-node3 failure-domain.beta.kubernetes.io/zone=qy
# kubectl get pods -l app=fortest
NAME READY STATUS RESTARTS AGE
testpod-7czf5 0/1 Pending 0 29m
testpod-hp6vc 1/1 Running 0 29m
testpod-tqkrc 0/1 Pending 0 29m
# kubectl label nodeskube-node2 failure-domain.beta.kubernetes.io/zone=qy
# kubectl get pods -l app=fortest
NAME READY STATUS RESTARTS AGE
testpod-7czf5 1/1 Running 0 29m
testpod-hp6vc 1/1 Running 0 29m
testpod-tqkrc 0/1 Pending 0 29m
# kubectl label nodes kube-node1failure-domain.beta.kubernetes.io/zone=xy
# kubectl get pods -l app=fortest
NAME READY STATUS RESTARTS AGE
testpod-7czf5 1/1 Running 0 29m
testpod-hp6vc 1/1 Running 0 29m
testpod-tqkrc 1/1 Running 0 29m
# kubectl get pods -l app=fortest -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
testpod-7czf5 1/1 Running 0 29m 10.244.1.42 kube-node2
testpod-hp6vc 1/1 Running 0 29m 10.244.119.177 kube-node3
testpod-tqkrc 1/1 Running 0 29m 10.244.0.11 kube-node1
对deployment busybox-httpd进行缩容，使得只有kube-node2上有相关pod，可以看到kube-node2和kube-node3上都有DaemonSet调度的pod处于Running状态
image.png
# kubectl get pods -l run=busybox-httpd -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox-httpd-7cf56cb476-dxnmj 1/1 Running 1 23h 10.244.1.25 kube-node2
#kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox-httpd-7cf56cb476-dxnmj 1/1 Running 1 23h 10.244.1.25 kube-node2
busybox-httpd-7cf56cb476-vq2c2 1/1 Running 1 23h 10.244.1.24 kube-node2
testpod-4tlp5 1/1 Running 0 9m1s 10.244.1.44 kube-node2
testpod-dlskr 1/1 Running 0 9m1s 10.244.119.179 kube-node3
testpod-mmxk7 0/1 Pending 0 9m1s <none> <none> <none> <none> 
