阿里云上实战openshift1-阿里ECS服务器上安装open
阿里云上购买ECS 3台:
其中1台ECS: 8核cpu, 内存16G, 系统盘80G, 作为openshift master以及企业dockerregistry
其中2台ECS：2核cpu, 内存8G, 系统盘30G、数据盘40G, 作为openshift node
【注】以下安装全部用root用户，root用户应有ssh远程登录权限
1.node节点安装
#设置主机名
hostnamectl set-hostname node1.honsen.com
hostnamectl set-hostname node2.honsen.com
#安装依赖包
yum install wget git net-tools bind-utils iptables-services bridge-utilsbash-completion kexec-tools sos psacct
yum update
#启用&启动Docker服务
systemctl enable docker; systemctl start docker
#修改docker配置
vi/etc/sysconfig/docker
首行Options追加上--insecure-registry172.30.0.0/16 --registry-mirror=https://docker.mirrors.ustc.edu.cn
#重启docker
systemctl restart docker
#启用&启动网络管理器
systemctl enable NetworkManager; systemctl start NetworkManager
#停止&禁用防火墙
systemctl stop firewalld ; systemctl disable firewalld
#Ansible和系统自带的urllib3有冲突, 卸载之: Error unpacking rpm package python-urllib3-1.10.2-3.el7.noarch
pip uninstall urllib3
#以下设置Docker中配置overlay/overlay2存储驱动
#检查overlay驱动是否已加载
lsmod | grep overlay
#以下设置node节点的docker数据存储区
see https://docs.openshift.org/latest/install_config/install/host_preparation.html#configuring-docker-storage
#查看磁盘情况
fdisk -l
#修改docker存储配置
vi /etc/sysconfig/docker-storage-setup
STORAGE_DRIVER=overlay2
DEVS=/dev/vdb
VG=docker-vg
#执行docker存储配置
docker-storage-setup
【注】执行这一步后会创建逻辑分区/dev/vdb1
fdisk -l
#格式化数据盘为xfs
mkfs.xfs -f /dev/vdb1
#停止docker
systemctl stop docker
rm-rf /var/lib/docker/*
#mount数据盘到/var/lib/docker
mount -t xfs /dev/vdb1 /var/lib/docker
#检查mount是否成功
df -Th /var/lib/docker
#配置重启时自动mount
vi /etc/fstab
加上一行：
/dev/vdb1 /var/lib/docker xfs defaults 0 0
#重启docker
systemctl restart docker
#查看docker配置信息
docker info
2.master安装
#设置主机名
hostnamectl set-hostname master.honsen.com
#设置本地域名解析
echo "172.18.113.134 master.honsen.com" >> /etc/hosts
echo "172.18.113.136 node1.honsen.com" >> /etc/hosts
echo "172.18.113.135 node2.honsen.com" >> /etc/hosts
#安装依赖包
yum install -y docker wget git net-tools bind-utils iptables-services bridge-utilsbash-completion
#启用&启动 Docker服务
systemctl enable docker; systemctl start docker
#修改docker配置
vi/etc/sysconfig/docker
首行Options追加上--insecure-registry172.30.0.0/16 --registry-mirror=https://docker.mirrors.ustc.edu.cn
#重启docker
systemctl restart docker
#启用&启动网络管理器
systemctl enable NetworkManager; systemctl start NetworkManager
#停止&禁用防火墙
systemctl stop firewalld ; systemctl disable firewalld
#Ansible和系统自带的urllib3有冲突, 卸载之: Error unpacking rpm package python-urllib3-1.10.2-3.el7.noarch
pip uninstall urllib3
#以下修改master的docker存储配置
vi /etc/sysconfig/docker-storage
DOCKER_STORAGE_OPTIONS=--storage-driver devicemapper --storage-optdm.fs=xfs
#重启docker
systemctl stop docker
rm -rf /var/lib/docker/*
systemctl restart docker
#查看docker配置信息
docker info
#安装, 启用&启动etcd分布式数据库
yum -y install etcd
systemctl enable etcd; systemctl start etcd
#下载安装EPEL
yum -y installhttps://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
#设置epelenable=0
sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo
#安装ansible
yum -y --enablerepo=epel install ansible pyOpenSSL
#生成秘钥, 实现ssh免密码连接
ssh-keygen -f ~/.ssh/id_rsa -N ''
#复制秘钥到集群中的所有节点,实现无密码访问
for host in master.honsen.com node1.honsen.com node2.honsen.com; do ssh-copy-id -i~/.ssh/id_rsa.pub -p root@$host; done
#设置ssh端口, 例如52113
vi ~/.ssh/config
host master.honsen.com
port 52113
host node1.honsen.com
port 52113
host node2.honsen.com
port 52113
#改变~/.ssh/config的权限
chmod 600 ~/.ssh/config
#测试ssh免密码连接
ssh node1.honsen.com
[honsen@node1 ~]$exit
#以下下载安装openshift-ansible
wget https://github.com/openshift/openshift-ansible/archive/openshift-ansible-3.7.31-1.tar.gz
tar zxvf openshift-ansible-3.7.31-1.tar.gz
vi /etc/ansible/hosts
[OSEv3:children]
masters
nodes
etcd
#Set variables common for all OSEv3 hosts
[OSEv3:vars]
#SSH user, this user should allow ssh based auth without requiring a password
ansible_ssh_user=root #ssh用户名
ansible_ssh_port=52113 #ssh服务端口
openshift_deployment_type=origin
openshift_release=3.7.1
#如果CPU内存满足条件, 可以注释掉openshift_disable_check
#Master节点要求 2 CPU核心, 16G内存, 40G磁盘
# Node节点要求 1 CPU核心, 8G内存, 20G磁盘
#openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability
#uncomment the following to enable htpasswd authentication; defaults toDenyAllPasswordIdentityProvider
openshift_master_identity_providers=[{'name':'htpasswd_auth','login':'true','challenge':'true','kind':'HTPasswdPasswordIdentityProvider','filename':'/etc/origin/master/htpasswd'}]
#host group for masters
[masters]
master.honsen.com
#host group for nodes, includes region info
[nodes]
master.honsen.com
node1.honsen.comopenshift_node_labels="{'region': 'primary', 'zone': 'default'}"
node2.honsen.comopenshift_node_labels="{'region': 'infra', 'zone': 'default'}"
[etcd]
master.honsen.com
#开始安装openshift集群(用ssh用户)
ansible-playbook ~/openshift-ansible-openshift-ansible-3.7.31-1/playbooks/byo/config.yml
3.成功安装后，检查安装结果
#基本检查
oc get nodes --show-labels
NAME STATUS AGE VERSION LABELS
master.honsen.com Ready,SchedulingDisabled 1h v1.7.6+a08f5eeb62 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=master.honsen.com,openshift-infra=apiserver
node1.honsen.com Ready 1h v1.7.6+a08f5eeb62 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node1.honsen.com,region=primary,zone=default
node2.honsen.com Ready 1h v1.7.6+a08f5eeb62 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node2.honsen.com,region=infra,zone=default
***安装过程中常见问题***
1) 多次执行ansible-playbook后，出现ssh error：
ssh: connect to host node1.honsen.com port 52113: No route to host\r\n
#检查防火墙状态
systemctl status firewalld
●firewalld.service
Loaded: masked (/dev/null; bad) //masked表明防火墙被锁住了！！！
Active: inactive (dead)
#以下是解决办法
#修改并重启iptabless
vi /etc/sysconfig/iptables
将
-AINPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
替换为
-AINPUT -p tcp -m state --state NEW -m tcp --dport 52113 -j ACCEPT
service iptables restart
2)Failure summary:
Hosts: master.honsen.com, node1.honsen.com, node2.honsen.com
Play: OpenShift Health Checks
Task: Run health checks(install) - EL
Message: One or more checksfailed
Details:
check"package_version":
Some required package(s) areavailable at a version
that is higher than requested
docker-1.13.1
This will prevent installing theversion you requested.
Please check your enabledrepositories or adjust openshift_release.
#解决办法
cd /roles/openshift_health_checker/openshift_checks
vi package_version.py
openshift_to_docker_version= {
(3,1): "1.8",
(3,2): "1.10",
(3,3): "1.10",
(3,4): "1.12",
(3,5): "1.12",
(3,6): "1.12",
(3,7): ["1.12", "1.13.1"], # <== here add the dockerversion that you want
(3,9): "1.13.1",
}
