Kubernetes 配置 coredns 作为集群内dns
2018-12-28 本文已影响0人
w也不知道
一、获取 yaml 文件
wget https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/coredns.yaml.sed
二、修改文件内容
# 主要修改以下 2 处内容
Corefile: |
.:53 {
errors
health
kubernetes cluster.local 10.0.0.0/24 {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
proxy . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
########################################################
# 10.0.0.0/24 是配置文件中 services 网段的地址，参数如下所示，怎么配置就怎么写
/opt/kubernetes/cfg/kube-apiserver:--service-cluster-ip-range=10.0.0.0/24 \
/opt/kubernetes/cfg/kube-controller-manager:--service-cluster-ip-range=10.0.0.0/24 \
/opt/kubernetes/cfg/kube-proxy:--cluster-cidr=10.0.0.0/24 \
# clusterIP 的值修改为 /opt/kubernetes/cfg/kubelet.config 中配置的 clusterDNS 的值
clusterIP=10.0.0.2
三、部署和测试
- 部署
kubectl apply -f coredns.yaml
- 测试成功结果如下所示
$ dig kubernetes.default.svc.cluster.local @10.0.0.2
; <<>> DiG 9.9.4-RedHat-9.9.4-72.el7 <<>> kubernetes.default.svc.cluster.local @10.0.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39810
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;kubernetes.default.svc.cluster.local. IN A
;; ANSWER SECTION:
kubernetes.default.svc.cluster.local. 5 IN A 10.0.0.1
;; Query time: 0 msec
;; SERVER: 10.0.0.2#53(10.0.0.2)
;; WHEN: Fri Dec 28 15:49:54 CST 2018
;; MSG SIZE rcvd: 117
四、遇到的问题
- kubernetes 节点每次重启后 coredns 失效，此时
ipvsadm命令显示结果如下：
$ ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP master:https rr
-> master:sun-sr-https Masq 1 0 0
TCP master:domain rr
- crondns pod 的日志显示如下
E1229 08:48:05.394899 1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:311: \
Failed to list *v1.Service: Get https://10.0.0.1:443/api/v1/services?limit=500&resourceVersion=0 \
: dial tcp 10.0.0.1:443: i/o timeout
- 解决办法，重启所有节点的
kube-proxy进程，删掉coredns并重新创建
# 注意一下命令先后顺序
kubectl delete -f coredns.yaml
systemctl restart kube-proxy # 所有节点
kubectl create -f coredns.yaml
- coredns 正常时
ipvsadm命令显示结果如下
$ ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP kubernetes.default.svc.clust rr
-> master:sun-sr-https Masq 1 0 0
TCP kube-dns.kube-system.svc.clu rr
-> 172.18.3.3:domain Masq 1 0 0
附录： 完整 coredns.yaml 文件内容
$ \cat coredns.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: coredns
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:coredns
rules:
- apiGroups:
- ""
resources:
- endpoints
- services
- pods
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:coredns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:coredns
subjects:
- kind: ServiceAccount
name: coredns
namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health
kubernetes cluster.local 10.0.0.0/24 {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
proxy . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/name: "CoreDNS"
spec:
replicas: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
spec:
serviceAccountName: coredns
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
nodeSelector:
beta.kubernetes.io/os: linux
containers:
- name: coredns
image: coredns/coredns:1.3.0
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
args: [ "-conf", "/etc/coredns/Corefile" ]
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
readOnly: true
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- all
readOnlyRootFilesystem: true
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
dnsPolicy: Default
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
---
apiVersion: v1
kind: Service
metadata:
name: kube-dns
namespace: kube-system
annotations:
prometheus.io/port: "9153"
prometheus.io/scrape: "true"
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "CoreDNS"
spec:
selector:
k8s-app: kube-dns
clusterIP: 10.0.0.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
protocol: TCP
- name: metrics
port: 9153
protocol: TCP
