CENTOS7 安装kubernetes集群笔记

环境准备

1. 确认官方推荐的centos版本和Docker版本官网链接
2. 关闭防火墙

• systemctl stop firewalld
• systemctl disable firewalld

3. 关闭swap

• swapoff -a
• vi /etc/fstab
  • 注释掉swap相关行，如下
  • #/dev/mapper/centos-swap
• cat /proc/swaps
  • 确认文件是空的

4. 关闭 SELinux

• setenforce 0
• sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

安装Docker

1. 更新相关依赖

• yum -y update && yum -y upgrade
• yum install yum-utils device-mapper-persistent-data lvm2

2. 添加Docker yum仓库(选阿里)

• 阿里
  • yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
• 官方
  • yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

3. 安装docker-ce(注意替换成官方推荐的版本)

• yum update && yum install docker-ce-18.06.2.ce

4. 配置docker daemon

• mkdir /etc/docker

      cat > /etc/docker/daemon.json <<EOF
      {
        "exec-opts": ["native.cgroupdriver=systemd"],
        "log-driver": "json-file",
        "log-opts": {
          "max-size": "100m"
        },
        "storage-driver": "overlay2",
        "storage-opts": [
          "overlay2.override_kernel_check=true"
        ]
      }
      EOF  
  

  • systemctl daemon-reload
  • systemctl restart docker
  • systemctl enable docker #开机启动

安装kubernetes

1. 添加kubernetes yum仓库(阿里)，官方仓库参考

    cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    [kubernetes]
    name=Kubernetes
    baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    enabled=1
    gpgcheck=0
    repo_gpgcheck=0
    gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
           http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF
   

2. 安装三套件

• yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
• systemctl enable --now kubelet

3. 修改k8s配置

    cat <<EOF >  /etc/sysctl.d/k8s.conf
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    net.ipv4.ip_forward = 1
    EOF
    sysctl --system
   

4. 偷梁换柱，绕过安装过程中从墙外google拉镜像的问题

   • 列出依赖镜像
     • kubeadm config images list
       • k8s.gcr.io/kube-apiserver:v1.12.2
       • k8s.gcr.io/kube-controller-manager:v1.12.2
       • k8s.gcr.io/kube-scheduler:v1.12.2
       • k8s.gcr.io/kube-proxy:v1.12.2
       • k8s.gcr.io/pause:3.1
       • k8s.gcr.io/etcd:3.2.24
       • k8s.gcr.io/coredns:1.2.2

• 下载阿里镜像，替换前缀

  • cat ./pull.sh

         for i in `kubeadm config images list`; do 
           imageName=${i#k8s.gcr.io/}
           docker pull registry.aliyuncs.com/google_containers/$imageName
           docker tag registry.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
           docker rmi registry.aliyuncs.com/google_containers/$imageName
         done;
    

  • sh pull.sh

5. 初始化集群

• kubeadm init --kubernetes-version=$(kubeadm version -o short) --pod-network-cidr=10.244.0.0/16 # pod-network-cidr=10.244.0.0/16 这个网段是之后安装flannel中需要且定死的

• 完成后输出

      Your Kubernetes control-plane has initialized successfully!
  
      To start using your cluster, you need to run the following as a regular user:
      
        mkdir -p $HOME/.kube
        sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
        sudo chown $(id -u):$(id -g) $HOME/.kube/config
      
      You should now deploy a pod network to the cluster.
      Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
        https://kubernetes.io/docs/concepts/cluster-administration/addons/
      
      Then you can join any number of worker nodes by running the following on each as root:
      
      kubeadm join 10.168.1.124:6443 --token idxunz.zj551qd8fxtnwv2g \
          --discovery-token-ca-cert-hash sha256:697ed215b32abd060d7902f2f588545c8ba0fd98478994e2814f71e31c777b9b
  

  • 这段最好保存，之后备用。

  • 如果kubeXXX命令运行错误，则执行第一段

      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    

6. 默认主节点不能部署，去除这一限制
   • kubectl taint nodes --all node-role.kubernetes.io/master-

安装网络插件flannel(这个容易上手)

1. 具体的连接可能会变化，参考官网

• kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml

加入节点

1. 新节点执行上面的操作直到安装完kubernets三套件。不需要执行初始化和安装flannel
2. 偷梁换柱(不知道为何主节点安装flannal时可以正常拉取)

• docker pull docker.io/mirrorgooglecontainers/pause:3.1

• docker tag docker.io/mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1

• docker rmi docker.io/mirrorgooglecontainers/pause:3.1

• docker pull quay-mirror.qiniu.com/coreos/flannel:v0.11.0-amd64

• docker tag quay-mirror.qiniu.com/coreos/flannel:v0.11.0-amd64 quay.io/coreos/flannel:v0.11.0-amd64

• docker rmi quay-mirror.qiniu.com/coreos/flannel:v0.11.0-amd64

• docker pull registry.aliyuncs.com/google_containers/kube-proxy:v1.16.2

• docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.16.2 k8s.gcr.io/kube-proxy:v1.16.2

• docker rmi registry.aliyuncs.com/google_containers/kube-proxy:v1.16.2

2. 把之前记录的命令拉出来执行

• kubeadm join 10.168.1.124:6443 --token idxunz.zj551qd8fxtnwv2g
  --discovery-token-ca-cert-hash sha256:697ed215b32abd060d7902f2f588545c8ba0fd98478994e2814f71e31c777b9b

3. 命令丢了？token失效过期了？

• 列出已经生成的token
  • kubeadm token list
• 重新生成token
  • kubeadm token create --print-join-command

如何查找问题，总有一款适合你

1. 查看节点状态

• kubectl get nodes

2. 查看服务和端口

• kubectl get services -A

3. 查看POD

• kubectl get pods -A

4. 查看pod配置(初始化错误也可以用这个查看)

• kubectl describe pod [PodName] --namespace=[PodNamespace]

5. 查看日志

• kubectl logs -f [PodName]

6. 下载image卡住了？重启下就好了

• systemctl restart kubelet

安装Dashboard

1. 改配置

• wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

• image换成阿里的

  • image: registry.cn-hangzhou.aliyuncs.com/kube_containers/kubernetes-dashboard-amd64

• 修改端口映射type改成NodePort, 增开nodePort

      apiVersion: v1
      metadata:
        labels:
          k8s-app: kubernetes-dashboard
        name: kubernetes-dashboard
        namespace: kube-system
      spec:
        type: NodePort
        ports:
          - port: 443
            targetPort: 8443
            nodePort: 31620
        selector:
          k8s-app: kubernetes-dashboard 
  

2. 安装

   • kubectl apply -f kubernetes-dashboard.yaml
   • kubectl replace --force -f kubernetes-dashboard.yaml#重装

3. 账户权限

   • 开账户
     • kubectl create serviceaccount k8sadmin -n kube-system
   • 赋权
     • kubectl create clusterrolebinding k8sadmin --clusterrole=cluster-admin --serviceaccount=kube-system:k8sadmin
   • 获得登录token
     • kubectl get secret -n kube-system
     • kubectl describe secret [TokenName] -n kube-system
     • 以上合并为一行命令
       • kubectl get secret -n kube-system | grep k8sadmin | cut -d " " -f1 | xargs -n 1 | xargs kubectl get secret -o 'jsonpath={.data.token}' -n kube-system | base64 --decode

4. 打开https://ip:port, 必须加上https, 忽略安全提示

5. 用token登录

安装WeaveScope

• 也需要改端口映射参考kubernetes-dashboard，或代理登录。
  • wget https://cloud.weave.works/k8s/scope.yaml?k8s-version=$(kubectl version | base64 | tr -d '\n') -O scope.yaml