kubeadm创建k8s集群v1.23.3(cilium网络插件

节点规划

准备工作

所有节点

modprobe  br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

所有节点，安装 containerd

apt install containerd -y
mkdir -p /etc/containerd/
containerd config default> /etc/containerd/config.toml

修改 /etc/containerd/config.toml
sandbox_image = "artifact.onwalk.net/k8s/pause:3.6"

然后重启服务 systemctl restart containerd

2. 安装1.20.2版本的 kubeadm、kubelet 和 kubectl

sudo apt-get install -y apt-transport-https ca-certificates curl -y
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

sudo cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

sudo apt-get update
sudo apt-get install -y kubelet=1.23.2-00 kubeadm=1.23.2-00 kubectl=1.23.2-00
sudo apt-mark hold kubelet kubeadm kubectl

3. config CRI

cat > /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF

systemctl restart containerd.service

初始化Master节点

• 方式一：

kubeadm init   \
--image-repository artifact.onwalk.net/k8s  \
--apiserver-advertise-address=10.60.116.31 \
--kubernetes-version v1.23.3  \
--service-cidr=192.168.0.0/16 \
--pod-network-cidr=172.16.0.0/12 \
--ignore-preflight-errors=all \
--skip-phases=addon/kube-proxy 

• 方式二：
  或者 kubeadm config print init-defaults > kubeadm-config.yaml 生成默认配置，修改部分参考如下(diff 输出结果)：

+  advertiseAddress: 10.60.116.31
+  criSocket: /run/containerd/containerd.sock
+imageRepository: artifact.onwalk.net/k8s
+kubernetesVersion: v1.23.3
+  dnsDomain: admin.local
+  serviceSubnet: 192.168.0.0/16
+  podSubnet: 172.16.0.0/12

最后执行命令： kubeadm init --skip-phases=addon/kube-proxy --config kubeadm-config.yaml （等效方式一，可以根据个人喜欢选择）详细配置参考
https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#hdr-Kubeadm_init_configuration_types

初始化Master节点完毕后，完成其他设置：
1 设置 kube-config

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

2. 自定义端口范围 修改 /etc/kubernetes/manifests/kube-apiserver.yaml

    - --service-node-port-range=1-65535

3. 去掉master节点污点 kubectl taint nodes --all node-role.kubernetes.io/master-

4. 安装 helm tools

wget https://mirrors.huaweicloud.com/helm/v3.5.2/helm-v3.5.2-linux-amd64.tar.gz 
tar -xf helm-v3.5.2-linux-amd64.tar.gz
mv linux-amd64/helm /usr/bin/
chmod 755 /usr/bin/helm

安装网络插件参考 helm安装Cilium https://www.jianshu.com/p/4ab7caab3bfa

Node节点

完成所有节点的初始化操作后，执行 Master 节点kubeadm token create --print-join-command 返回的命令:

参考

https://github.com/linkerd/linkerd2/issues/5932
https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

K8S V1.23 版本镜像列表

docker pull k8s.gcr.io/pause:3.6
docker pull k8s.gcr.io/kube-apiserver:v1.23.3
docker pull k8s.gcr.io/kube-controller-manager:v1.23.3
docker pull k8s.gcr.io/kube-scheduler:v1.23.3
docker pull k8s.gcr.io/kube-proxy:v1.23.3
docker pull k8s.gcr.io/coredns/coredns:v1.8.6
docker pull k8s.gcr.io/etcd:3.5.1-0

docker tag k8s.gcr.io/etcd:3.5.1-0 artifact.onwalk.net/k8s/etcd:3.5.1-0
docker tag k8s.gcr.io/pause:3.6 artifact.onwalk.net/k8s/pause:3.6
docker tag k8s.gcr.io/coredns/coredns:v1.8.6 artifact.onwalk.net/k8s/coredns:v1.8.6
docker tag k8s.gcr.io/kube-apiserver:v1.23.3  artifact.onwalk.net/k8s/kube-apiserver:v1.23.3
docker tag k8s.gcr.io/kube-controller-manager:v1.23.3  artifact.onwalk.net/k8s/kube-controller-manager:v1.23.3
docker tag k8s.gcr.io/kube-scheduler:v1.23.3  artifact.onwalk.net/k8s/kube-scheduler:v1.23.3
docker tag k8s.gcr.io/kube-proxy:v1.23.3  artifact.onwalk.net/k8s/kube-proxy:v1.23.3

docker push artifact.onwalk.net/k8s/pause:3.6
docker push artifact.onwalk.net/k8s/kube-apiserver:v1.23.3
docker push artifact.onwalk.net/k8s/kube-controller-manager:v1.23.3
docker push artifact.onwalk.net/k8s/kube-scheduler:v1.23.3
docker push artifact.onwalk.net/k8s/kube-proxy:v1.23.3
docker push artifact.onwalk.net/k8s/coredns:v1.8.6
docker push artifact.onwalk.net/k8s/etcd:3.5.1-0