2. LVS命令使用
2021-03-13 本文已影响0人
随便写写咯
3.3 LVS相关软件
3.3.1 程序包：ipvsadm
Unit File: ipvsadm.service
主程序：/usr/sbin/ipvsadm
规则保存工具：/usr/sbin/ipvsadm-save
规则重载工具：/usr/sbin/ipvsadm-restore
配置文件：/etc/sysconfig/ipvsadm-config
ipvs调度规则文件：/etc/sysconfig/ipvsadm
3.3.2 ipvsadm 命令
ipvsadm核心功能：
集群服务管理：增、删、改
集群服务的RS管理：增、删、改
查看
ipvsadm工具的使用:
ipvsadm安装
[16:35:51 root@lvs ~]#yum -y install ipvsadm
[Service]
Type=oneshot
ExecStart=/bin/bash -c "exec /sbin/ipvsadm-restore < /etc/sysconfig/ipvsadm" # 启动ipvs时, 会加载 /etc/sysconfig/ipvsadm中保存的规则
ExecStop=/bin/bash -c "exec /sbin/ipvsadm-save -n > /etc/sysconfig/ipvsadm" # 关闭ipvs时, 会保存内存中的ipvs规则到 /etc/sysconfig/ipvsadm文件
ExecStop=/sbin/ipvsadm -C # 并且执行/sbin/ipvsadm -C, 清空所有lvs规则
RemainAfterExit=yes
命令帮助
Commands:
Either long or short options are allowed.
--add-service -A add virtual service with options # virtual service就是一个lvs集群
--edit-service -E edit virtual service with options
--delete-service -D delete virtual service
--clear -C clear the whole table
--restore -R restore rules from stdin
--save -S save rules to stdout
--add-server -a add real server with options # 给集群添加RS
--edit-server -e edit real server with options
--delete-server -d delete real server
--list -L|-l list the table
--zero -Z zero counters in a service or all services
virtual-service:
--tcp-service|-t service-address service-address is host[:port] # 让lvs处理tcp协议请求, 如http
--udp-service|-u service-address service-address is host[:port] # 让lvs处理udp协议请求
--sctp-service service-address service-address is host[:port]
--fwmark-service|-f fwmark fwmark is an integer greater than zero # f 防火墙标签
scheduler:
--scheduler -s scheduler one of rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq|fo|ovf|mh,
the default scheduler is wlc.
管理集群服务：增、改、删
增、修改：
ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p [timeout]]
删除：
ipvsadm -D -t|u|f service-address
service-address：
-t|u|f：
-t: TCP协议的端口，VIP:TCP_PORT
-u: UDP协议的端口，VIP:UDP_PORT
-f：firewall MARK，标记，一个数字
[-s scheduler]：指定集群的调度算法，默认为wlc
管理集群上的RS：增、改、删
增、改：
ipvsadm -a|e -t|u|f service-address -r server-address [-g|i|m] [-w weight]
删：
ipvsadm -d -t|u|f service-address -r server-address
server-address：
rip[:port] 如省略port，不作端口映射
选项：
lvs类型：
-g: gateway, dr类型，默认
-i: ipip, tun类型
-m: masquerade, nat类型
-w weight：权重
创建一个LVS集群
[16:54:22 root@lvs ~]#ipvsadm -A -t 10.0.0.100:80
查看集群
[16:54:27 root@lvs ~]#ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:http wlc
[16:54:30 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wlc
向集群添加Real Server
#默认DR模型
#-g:DR模型
#-i: tunnel模型
#-m: NAT模型
[16:55:19 root@lvs ~]#ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.101 -w 2
[16:58:40 root@lvs ~]#ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.102 -w 3
[16:58:44 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wlc
-> 10.0.0.101:80 Route(表示DR模型) 2 0 0
-> 10.0.0.102:80 Route(表示DR模型) 3 0 0
删除RS
[16:58:47 root@lvs ~]#ipvsadm -d -t 10.0.0.100:80 -r 10.0.0.101
[16:59:57 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wlc
-> 10.0.0.102:80 Route 3 0 0
删除集群
[17:00:02 root@lvs ~]#ipvsadm -D -t 10.0.0.100:80
[17:00:17 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
修改集群
[17:00:21 root@lvs ~]#ipvsadm -A -t 10.0.0.100:80 -s wrr
[17:00:58 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
[17:01:24 root@lvs ~]#ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.101 -w 3
[17:01:58 root@lvs ~]#ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.102 -w 5
[17:02:01 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
保存规则
#默认会打印到标准输出
#-n, 将服务解析成数字, 不加-n会作自动解析, 比较耗时
[17:02:04 root@lvs ~]#ipvsadm -S -n
-A -t 10.0.0.100:80 -s wrr
-a -t 10.0.0.100:80 -r 10.0.0.101:80 -g -w 3
-a -t 10.0.0.100:80 -r 10.0.0.102:80 -g -w 5
#ipvsadm-save命令是一个Shell脚本, 会调用ipvsadm -S命令, 也是将规则默认打印到标准输出
[14:08:52 root@vip-1 ~]#ipvsadm-save
[17:03:29 root@lvs ~]#ipvsadm-save
-A -t 10.0.0.100:http -s wrr
-a -t 10.0.0.100:http -r 10.0.0.101:http -g -w 3
-a -t 10.0.0.100:http -r 10.0.0.102:http -g -w 5
[17:03:33 root@lvs ~]#ipvsadm-save -n
-A -t 10.0.0.100:80 -s wrr
-a -t 10.0.0.100:80 -r 10.0.0.101:80 -g -w 3
-a -t 10.0.0.100:80 -r 10.0.0.102:80 -g -w 5
#将规则保存到文件里
[17:03:42 root@lvs ~]#ipvsadm-save > /data/ipvsadm.rule
[17:05:09 root@lvs ~]#cat /data/ipvsadm.rule
-A -t 10.0.0.100:http -s wrr
-a -t 10.0.0.100:http -r 10.0.0.101:http -g -w 3
-a -t 10.0.0.100:http -r 10.0.0.102:http -g -w 5
清空集群中的规则
[17:05:12 root@lvs ~]#ipvsadm -C
[17:05:39 root@lvs ~]#ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
加载保存的ipvsadm规则
#ipvsadm-restore也是一个Shell脚本, 会调用ipvsadm -R,默认从标准输入读取规则, 可以接文件
[17:05:41 root@lvs ~]#ipvsadm-restore < /data/ipvsadm.rule
[17:06:02 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
#测试ipvsadm -R
[17:06:05 root@lvs ~]#ipvsadm -C
[17:07:20 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[17:07:23 root@lvs ~]#ipvsadm -R < /data/ipvsadm.rule
[17:07:29 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
将规则保存到能开启服务自动加载的文件里 /etc/sysconfig/ipvsadm
[14:17:02 root@vip-1 ~]#ls /etc/sysconfig/ip*
/etc/sysconfig/ip6tables-config /etc/sysconfig/iptables-config /etc/sysconfig/ipvsadm-config
/etc/sysconfig/ipvsadm默认是不存在的, 因此无法直接启动服务, 因为ipvsadm的service文件, 启动时会读取该文件内容
[17:09:08 root@lvs ~]#systemctl start ipvsadm
Job for ipvsadm.service failed because the control process exited with error code.
See "systemctl status ipvsadm.service" and "journalctl -xe" for details.
因此, 需要手动创建该文件, 并且, 即使该文件内容为空, 也无所谓, 只不过是启动服务后是没有任何规则的
[17:09:08 root@lvs ~]#touch /etc/sysconfig/ipvsadm
[17:11:15 root@lvs ~]#systemctl status ipvsadm
● ipvsadm.service - Initialise the Linux Virtual Server
Loaded: loaded (/usr/lib/systemd/system/ipvsadm.service; disabled; vendor preset: disabled)
Active: inactive (dead)
[17:11:24 root@lvs ~]#systemctl start ipvsadm
[17:11:31 root@lvs ~]#systemctl status ipvsadm
● ipvsadm.service - Initialise the Linux Virtual Server
Loaded: loaded (/usr/lib/systemd/system/ipvsadm.service; disabled; vendor preset: disabled)
Active: active (exited) since Sat 2021-03-13 17:11:31 CST; 7s ago
Process: 1558 ExecStart=/bin/bash -c exec /sbin/ipvsadm-restore < /etc/sysconfig/ipvsadm (code=exited, status=0/SUCCESS)
Main PID: 1558 (code=exited, status=0/SUCCESS)
Mar 13 17:11:31 lvs systemd[1]: Starting Initialise the Linux Virtual Server...
Mar 13 17:11:31 lvs systemd[1]: Started Initialise the Linux Virtual Server.
[14:21:06 root@vip-1 ~]#ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
将先前保存到/data/ipvsadm.rule文件中的规则导入内存中
[17:12:00 root@lvs ~]#ipvsadm -R < /data/ipvsadm.rule
[17:12:16 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
停止ipvsadm服务, 测试加载的规则会被自动保存到/etc/sysconfig/ipvsadm.service文件中
[17:12:20 root@lvs ~]#cat /etc/sysconfig/ipvsadm
[17:13:08 root@lvs ~]#systemctl stop ipvsadm
[17:13:12 root@lvs ~]#cat /etc/sysconfig/ipvsadm
-A -t 10.0.0.100:80 -s wrr
-a -t 10.0.0.100:80 -r 10.0.0.101:80 -g -w 3
-a -t 10.0.0.100:80 -r 10.0.0.102:80 -g -w 5
停止服务后, 内存中的规则也会被清空
[17:13:13 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
注意:
- ipvsadm开启服务, 会自动读取/etc/sysconfig/ipvsadm中定义的规则, 如果该文件为空,那就是没有规则
- ipvsadm关闭服务, 会先把加载到内存中的规则, 保存到/etc/sysconfig/ipvsadm文件中, 然后会执行ipvsadm -C把加载到内存中的规则情况
- 这些都是在Service文件中定义的
再次启动ipvsadm, 验证ipvsadm会从/etc/sysconfig/ipvsadm文件中读取规则
[17:13:24 root@lvs ~]#systemctl start ipvsadm
[17:15:44 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
验证服务开机自动启动会加载规则
[17:15:49 root@lvs ~]#systemctl enable --now ipvsadm
Created symlink /etc/systemd/system/multi-user.target.wants/ipvsadm.service → /usr/lib/systemd/system/ipvsadm.service.
[17:16:01 root@lvs ~]#reboot
a[17:17:23 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0
-> 10.0.0.102:80 Route 5 0 0
补充:
加载到内存的ipvsadm的规则会存在/proc/net/ip_vs文件里, 以16进制数字显示
[17:18:07 root@lvs ~]#cat /proc/net/ip_vs
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 0A000064:0050 wrr
-> 0A000066:0050 Route 5 0 0
-> 0A000065:0050 Route 3 0 0
清空定义的所有内容
ipvsadm -C
清空计数器
ipvsadm -Z [-t|u|f service-address]
[17:18:10 root@lvs ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 10.0.0.101:80 Route 3 0 0 # 活动链接和非活动链接数会以计数器显示, 可以用 ipvsadm -Z 清空
-> 10.0.0.102:80 Route 5 0 0
查看
ipvsadm -L|l [options]
–numeric, -n：以数字形式输出地址和端口号
–exact：扩展信息，精确值
–connection，-c：当前IPVS连接输出
–stats：统计信息
–rate ：输出速率信息
[17:19:52 root@lvs ~]#ipvsadm -Lnc
IPVS connection entries
pro expire state source virtual destination
