双主模型keepalived高可用集群
2017-09-08 本文已影响0人
ALuckyLive
1、双主模型ipvs(keepalived) 高可用集群
2、双主模型nginx(keepalived)高可用集群
1、准备五个虚拟机(centos7)；
note1、note2、note3、note4、client
2、配置静态ip地址，分别为；
note1;172.16.253.10
note2;172.16.253.11
note3;172.16.253.13
note4;172.16.253.14
3、将防火墙及selinux关闭；
#systemctl stop firewalld
#systemctl disable firewalld
#systemctl is-enabled firewalld
#setenforce 0
#vim /etc/selinux/config
SELINUX=permissive
#getenforce
4、配置yum源；
5、安装keepalived、ipvsadm、nginx、httpd；
6、同步时间；
vim /etc/chrony
server 172.16.0.1 iburst
systemctl start chronyd.service
chronyc sources---查看同步状态
7、设置/etc/hosts文件，做ip地址解析；
ipvs(keepalived)
1、note1、note2，设置单主模式keepalived
配置/etc/keepalived/keepalived.conf文件；
设置全局配置、vrrp_instance虚拟路由器配置
[root@note1 ~]# cat keepalived.conf.shuangzhu
! Configuration File for keepalived
global_defs {
notification_email {
keepalived@note.com
}
notification_email_from keepalived_admin@note.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id note1
vrrp_mcast_group4 224.12.0.18
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
将note1的此配置文件拷贝到note2上，将配置文件MASTER改为BACKUP(主用改为备用)、将prioirty权重改为95；
[root@note1 ~]#systemctl start keepalived
#ip a l
#mail
[root@note2 ~]#systemctl start keepalived
#ip a l
#mail
将note1主用keepalived停用，查看note2备用keepalived是否升为主用；
2、note1、note2，设置双主模式keepalived；
#yum install keepalived
在/etc/keepalived/目录下创建一个脚本,并调用，生成邮件；
#!/bin/bash
contact='root@localhost'
notify() {
local mailsubject="$(hostname) to be $1, vip floating"
local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
将原有的配置文件中的vrrp_instance配置段各复制一份，进行修改。如果主备模式三台可以复制三分，并进行设置；
1、将虚拟路由器名称VI_1改为VI_2。
2、设置虚拟路由器id，11改为12。
3、更改身份验证auto_PASS。
4、更改主备设置及权重。
note1配置文件；
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 12
priority 95
advert_int 1
authentication {
auth_type PASS
auth_pass yKt5PsOZ
}
virtual_ipaddress {
172.16.253.102/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
#双主模式；有两个虚拟路由器VI_1和VI_2，两个虚拟路由器有各自的ip地址；
*note1; VI_1 为MASTER，VI_2为BACKUP
*note2; VI_2 为MASTER，VI_1为BACKUP
测试；tcp -nn -l ens34 host 244.12.0.18
systemctl stop keepalived
tcp -nn -l ens34 host 244.12.0.18
3、note3、note4，设置网页index.html文件；
#yum install nginx
1、将默认网页文件内容删除重新编辑；
note3；vim /usr/share/nginx/html/index.html
<h1>RS1</h1>
note4；vim /usr.share/nginx/html/index.html
<h1>RS2</h1>
4、note3、note4；将虚拟路由器ip添加到回环网卡lo上；
note3;
编辑一个脚本并执行；
vim setrs.sh
#!/bin/bash
vip=172.16.253.101
mask=255.255.255.255
iface="lo:0"
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig $iface $vip netmask $mask broadcast $vip up
route add -host $vip dev $iface
;;
stop)
ifconfig $iface down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
检查语法；bash -n setrs.sh
检查语法并执行；bash -x setrs.sh start
执行成功后将虚拟路由器ip添加到lo网卡上;
将脚本ip改为172.16.253.102，在lo网卡上添加第二个虚拟路由器ip;
note4;
根据note3步骤将虚拟路由器ip添加到note4后端服务器lo网卡上;
5、note1、note2，设置ipvs相关的vs、rs---地址；
1、编辑/etc/keepalived/keepalived.conf文件，设置virtual_server 的vs及rs的地址；
virtual_server 172.16.253.102 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.16.253.13 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.253.14 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
note1;在各自的虚拟路由器段，如VI_1、VI_2的vrrp配置段下添加virtual_server段，注意虚拟服务器地址，分别为；172.16.253.101,172.16.253.102.
note2;根据note1配置。
6、note1,note2;在配置文件/etc/keepalived/keepaived.conf，virtual-server板块中 sorry_server错误界面选项；
virtual_server 172.16.253.101 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.16.253.13 80 {
weight 1
HTTP_GET {
安装ngnix或httpd,将默认网页文件改为自己所需要的内容；
[root@note1 ~]# vim /usr/share/nginx/html/index.html
<h1>sorry from Director1</h1>
note1,note2都设置此项；
7、client测试高可用性；
访问ip地址；172.16.253.101或172.16.253.101
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
当将两台后端主机RS服务器停掉，在访问172.16.253.101或172.16.253.102；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
<h1>sorry from Director1</h1>
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.102;done
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
<h1>sorry from Director2</h1>
nginx（keepalived）
1、实现单主模型下nginx（keepalived）调用后端RS服务器；
1、启用nginx调用功能，编写配置文件/etc/nginx/nginx.conf；
note1,note2设置配置文件；
upstream websrvs {
server 172.16.253.13:80
server 172.16.253.14:80
}
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://websrvs;
}
设置完成后，访问172.16.253.10测试一下；
[root@note1 ~]# curl http://172.16.253.10
<h1>RS1</h1>
[root@note1 ~]# curl http://172.16.253.10
<h1>RS2</h1>
2、在/etc/keepalived/keepalived.conf文件中调用命令段，实现降权、升权的功能，主备切换的功能；
设置调用脚本或程需命令选项；
查看示例；
[root@note1 ~]# grep -i vrrp_script /usr/share/doc/keepalived-1.2.13/samples/*
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_sshd {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_haproxy {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_http_port {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_https_port {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_smtp_port {
[root@note1 ~]# cat /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck
! Configuration File for keepalived
vrrp_script chk_sshd {
script "killall -0 sshd" # cheaper than pidof
interval 2 # check every 2 seconds
weight -4 # default prio: -4 if KO
fall 2 # require 2 failures for KO
rise 2 # require 2 successes for OK
}
设置手动制造故障及设置调用nginx脚本；
global_defs {
notification_email {
keepalived@note.com
}
notification_email_from keepalived_admin@note.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id note1
vrrp_mcast_group4 224.12.0.18
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]]" && exit 1 || exit 0"
interval 1
weight -10
fall 2
rise 2
}
vrrp_script chk_nginx {
skript "killall -0 nginx"
interval 2
weight -10
fall 2
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
track_script{
chk_nginx
chk_down
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
3、将配置文件复制到note2上并进行修改；
将MASTER改为BACKUP,单主模式配置完成。
进行访问测试；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
[root@note1 ~]# cd /etc/keepalived/
[root@note1 keepalived]# touch down
[root@note1 ~]# tcpdump -i ens34 -nn host 224.12.0.18
11:01:41.580252 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:41.580881 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:01:41.581866 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:42.583792 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:43.587375 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype
2、实现双主模式下nginx ( keepalived ) 调用后端RS服务器；
1、note1; 在配置文件/etc/keepalived/keepalived.conf中, 添加复制一份
virtual_server区域内容，并进行修改。MASTER,BACKUP、权、验证码、虚拟路由器ip(vip)
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt5PsOZ
}
virtual_ipaddress {
172.16.253.102/16 dev ens34
}
track_script{
chk_nginx
chk_down
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
note2；按note1的步骤配置、/etc/keepalived/keepalived.conf文件；
2、cilent进行测试，也可在note1和note2端使用touch down来进行检测；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.102;done
<h1>RS1</h1>
<h1>RS2</h1>
<h1>RS1</h1>
<h1>RS2</h1>
[root@note1 keepalived]# touch down
[root@note1 ~]# tcpdump -i ens34 -nn host 224.12.0.18
11:21:53.974338 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:53.976207 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:54.976431 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:54.978223 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:55.978463 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:55.980380 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:56.980566 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:56.982511 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 90, authtype simple, intvl 1s, length 20
11:21:56.986256 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:56.987810 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 90, authtype simple, intvl 1s, length 20
11:21:56.995422 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:57.982627 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:57.998655 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:58.984764 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:59.000745 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:59.986896 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
3、测试故障修复抢占模式；
note1；/etc/keepalived/keepalied.conf目录中的调用了notify脚本文件，在脚本文件中添加一项内容；systemctl start nginx
在notify backup下添加一行；systemctl start nginx
