Kubernetes 安装、初始化、pod 网络测试
2023-02-09 本文已影响0人
河码匠
基于
kubeadm=1.25.0 kubectl=1.25.0 kubelet=1.25.0进行试验安装。1.24 以后的安装方式相同
| 主机名 | 操作系统 | IP | 配置 |
|---|---|---|---|
| master | ubuntu 20.04 | 192.168.17.130 | 4C 4M 50G |
| node1 | ubuntu 20.04 | 192.168.17.131 | 4C 4M 50G |
| node2 | ubuntu 20.04 | 192.168.17.132 | 4C 4M 50G |
一、安装前准备
所有节点都执行
1. 修改主机名
hostnamectl set-hostname xxx | bash
2. 修改 hosts 文件
root@master:~/k8s# vim /etc/hosts
192.168.17.130 master
192.168.17.131 node1
192.168.17.132 node2
3. 配置 ssh 免密登录
root@master:~/k8s# ssh-keygen -t rsa
将 ssh token 复制到所有节点
root@master:~/k8s# ssh-copy-id xxxx
4. 加载网络插件且修改配置
modprobe br_netfilter
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl -p /etc/sysctl.d/k8s.conf
5. 关闭交换分区 swappoff
swappof -a
注释下面文件中的 swapp 部分内容
vim /etc/fstab
查看是否关闭成功
free -h
6. 关闭防火墙
ufw disable
查看防火墙状态
ufw status
7. 修改 apt-get 源。查看
二、安装 containerd
所有节点都执行
1. 删除本机存在的docker相关内容
sudo apt-get remove docker docker-engine docker.io containerd runc
2. 安装需要的包
apt-get update
apt-get install ca-certificates curl gnupg
3. 添加docker key文件
mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
4. 添加 docker 源
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
5. 更新 apt
apt-get update
6. 安装 containerd 并锁定版本
apt-get install -y containerd.io=1.6.6-1
apt-mark hold containerd.io=1.6.6-1
7. 生成 containerd 配置文件
-
创建
/etc/containerd文件夹
mkdir -p /etc/containerd
-
生成 containerd 配置文件
containerd config default > /etc/containerd/config.toml
8. 修改 containerd 配置文件
-
开启 Systemd
SystemdCgroup 改为 true
-
修改
sandbox_image镜像仓位置
pause:3.7 版本根据 kubernetes 而定
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7"
9. 启动 containerd 并开机启动
systemctl enable containerd --now
10. 修改 /etc/crictl.yaml
设置 kubernetes 创建 pod 时的运行时使用 containerd
cat > /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF
11. 安装 docker 方便管理镜像
-
安装 docker 并设置开机启动
apt-get install docker-ce
systemctl enable docker --now
-
修改镜像加速
root@master:~/k8s# vim /etc/containerd/config.toml
config_path = "/etc/containerd/certs.d"
root@master:~/k8s# mkdir /etc/containerd/certs.d/docker.io/ -p
-
配置加速器地址
root@master:~/k8s# vim /etc/containerd/certs.d/docker.io/hosts.toml
[host."https://xxxx.mirror.aliyuncs.com",host."https://registry.docker-cn.com"]
capabilities = ["pull"]
https://xxxx.mirror.aliyuncs.com 这是阿里的镜像加速器登录后可以看到
-
docker 镜像加速器
root@master:~/k8s# mkdir /etc/docker
root@master:~/k8s# vim /etc/docker/daemon.json
{
"registry-mirrors":["https://xxxx.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com"]
}
-
查看 docker 镜像源
docker info|grep Mirrors -A 1
三、安装 kubernetes
1，2，3步骤节点都执行。其他步骤 master 节点执行
1. 配置 kubernetes 源
tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main
EOF
apt-get update
2. 安装 kubeadm kubectl kubelet
apt-get install -y kubeadm=1.25.0-00 kubectl=1.25.0-00 kubelet=1.25.0-00
3. 设置容器运行时
crictl config runtime-endpoint /run/containerd/containerd.sock
4. 导出初始化使用的 yaml
kubeadm config print init-defaults > kubeadm.yaml
-
yaml 说明
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
# 控制节点 IP
advertiseAddress: 192.168.17.130
bindPort: 6443
nodeRegistration:
# 使用的容器运行时的位置
# 如果本机安装的 docker 把 criScoket 删除，kubernetes 会找系统默认。
# 但是 1.24 以后的版本直接使用 containerd。继续使用 docker 安装 kubernetes 会有问题
# 这里使用自定义的容器运行时位置 /run/containerd/containerd.sock
criSocket: unix:///run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
# 控制节点主机名
name: master
taints: null
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
# 镜像仓库这里 阿里云
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
# kubernetes 版本号。根据自己安装的定义
kubernetesVersion: 1.25.0
networking:
dnsDomain: cluster.local
# service 子网
serviceSubnet: 10.96.0.0/12
# pod 子网
podSubnet: 10.244.0.0/16
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
# KubeProxyConfiguration 代理模式，指定 ipvs，默认是 iptables，iptables 效率低。
kind: KubeProxyConfiguration
mode: ipvs
---
apiVersion: kubelet.config.k8s.io/v1beta1
# 修改 KubeletConfiguration 驱动为 systemd
kind: KubeletConfiguration
cgroupDriver: systemd
5. 初始化 kubernetes 集群
kubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification
-
添加
kubectl配置文件。安装成功后会提示执行下面的命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
6. 添加 node 节点
-
master 执行，生成添加节点命令和 token
root@master:~/k8s# kubeadm token create --print-join-command
kubeadm join 192.168.17.130:6443 --token v1d2bf.zkg38gi5wdktbb7d --discovery-token-ca-cert-hash sha256:8615f5a5d47405c977837c3f1088e67f91b754bc5812f599fe805848c6030db2
-
node 节点执行
kubeadm join 192.168.17.130:6443 --token v1d2bf.zkg38gi5wdktbb7d --discovery-token-ca-cert-hash sha256:8615f5a5d47405c977837c3f1088e67f91b754bc5812f599fe805848c6030db2
-
添加成功会提示在 master 节点查看 node
kubectl get nodes
7. 修改节点标签
kubectl label nodes xxx node-role.kubernetes.io/work=work
-
查看 node。发现节点 NotReady 是因为没有安装网络插件
root@master:~/k8s# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 6m56s v1.25.0
node1 NotReady work 2m54s v1.25.0
node2 NotReady work 117s v1.25.0
-
查看 pod 详情发现
coredns是pending状态，因为没有网络
root@master:~/k8s# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-7f8cbcb969-9j5v6 0/1 Pending 0 9m38s <none> <none> <none> <none>
coredns-7f8cbcb969-vnz5h 0/1 Pending 0 9m38s <none> <none> <none> <none>
etcd-master 1/1 Running 0 10m 192.168.17.130 master <none> <none>
kube-apiserver-master 1/1 Running 0 10m 192.168.17.130 master <none> <none>
kube-controller-manager-master 1/1 Running 0 10m 192.168.17.130 master <none> <none>
kube-proxy-5hv9x 1/1 Running 0 9m39s 192.168.17.130 master <none> <none>
kube-proxy-gv5g8 1/1 Running 0 5m21s 192.168.17.132 node2 <none> <none>
kube-proxy-smk2m 1/1 Running 0 6m18s 192.168.17.131 node1 <none> <none>
kube-scheduler-master 1/1 Running 0 10m 192.168.17.130 master <none> <none>
8. 安装网络插件
下载 calico.yaml 文件 官网地址 根据自己的版本下载 yaml
kubectl apply -f calico.yaml
：如果试验环境是单网卡则不用修改配置，如果对多网卡需要在配置文件中添加下面配置
kind: DaemonSet
......
containers:
- name: calico-node
......
env:
......
name: IP_AUTODETECTION_METHOD
valule: "interface=eth0" #这里的 eth0 是有网络的网卡
-
再次查看 pods
root@master:~/k8s# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-6744f6b6d5-hjwqp 1/1 Running 0 6m45s
kube-system calico-node-9fqfg 1/1 Running 0 6m45s
kube-system calico-node-hv2tf 1/1 Running 0 6m45s
kube-system calico-node-ts6hs 1/1 Running 0 6m45s
kube-system coredns-7f8cbcb969-9j5v6 1/1 Running 0 19m
kube-system coredns-7f8cbcb969-vnz5h 1/1 Running 0 19m
kube-system etcd-master 1/1 Running 0 20m
kube-system kube-apiserver-master 1/1 Running 0 20m
kube-system kube-controller-manager-master 1/1 Running 0 20m
kube-system kube-proxy-5hv9x 1/1 Running 0 19m
kube-system kube-proxy-gv5g8 1/1 Running 0 15m
kube-system kube-proxy-smk2m 1/1 Running 0 16m
kube-system kube-scheduler-master 1/1 Running 0 20m
-
再次查看那 nodes
root@master:~/k8s# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane 20m v1.25.0
node1 Ready work 16m v1.25.0
node2 Ready work 15m v1.25.0
三、测试 pod 中网络
1. 在 master 节点创建一个 pod
kubectl run busybox --image docker.io/library/busybox:1.28 --image-pull-policy=IfNotPresent --restart=Never --rm -it busybox -- sh
2. 创建成功后会自动进入 pod ，执行 ping
/ # ping baidu.com
PING baidu.com (110.242.68.66): 56 data bytes
64 bytes from 110.242.68.66: seq=0 ttl=127 time=26.034 ms
64 bytes from 110.242.68.66: seq=1 ttl=127 time=45.559 ms
3. 测试 DNS
/ # nslookup kubernetes.default.svc.cluster.local
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default.svc.cluster.local
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
