让虚拟机支持高可用的vip
2017-02-20 本文已影响199人
魔哈Moha
背景####
线上部分虚拟机业务还是使用传统基于keepalived的方式做高可用方案，所以在配置vip的时候实际上这个ip是被隔离的。
我们都知道，Neutron在处理虚拟机地址时是绑定IP和MAC地址，一方面是为了防止ARP欺骗，另一方面也是杜绝用户在虚拟机里面私建IP地址。所以直接在虚拟机上绑定一个VIP是不能被访问的，OpenStack好像是在Havana版本上release了支持vrrp协议的功能。
操作####
- 创建一个vip
$ neutron port-create --fixed-ip ip_address=172.16.1.40 --security-group default
Created a new network:
+-----------------+--------------------------------------+
| Field | Value |
+-----------------+--------------------------------------+
| admin_state_up | True |
| id | 7bb20853-32fe-4e8c-8969-b011c612205d |
| mtu | 0 |
| name | mynet |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tenant_id | fc74c6ec4f3f48eb9a9c6526c36b297a |
+-----------------+--------------------------------------+
- 绑定给需要vip的虚拟机
$ neutron port-list
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| 40e7b1f7-1127-408f-9460-fbfb3f0acc74 | | fa:16:3e:23:97:8c | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.3"} |
| 44d2cad8-1762-4d23-aaab-06da89514119 | | fa:16:3e:ed:da:20 | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.254"} |
| 7bb20853-32fe-4e8c-8969-b011c612205d | | fa:16:3e:6f:29:c3 | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.40"} |
| c4283773-a67c-439f-b0cd-df0dcac6e531 | | fa:16:3e:84:9b:d7 | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.1"} |
| f7364078-7ba6-4383-ae8b-68323eada711 | | fa:16:3e:04:7a:5b | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.2"} |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
$ neutron port-update f7364078-7ba6-4383-ae8b-68323eada711 --allowed_address_pairs list=true type=dict ip_address=172.16.1.40
$ neutron port-update 40e7b1f7-1127-408f-9460-fbfb3f0acc74 --allowed_address_pairs list=true type=dict ip_address=172.16.1.40
$ neutron port-show 40e7b1f7-1127-408f-9460-fbfb3f0acc74
+-----------------------+-----------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | {"ip_address": "172.16.1.40", "mac_address": "fa:16:3e:23:97:8c"} |
| binding:host_id | mitaka.dev.jumei.com |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2017-02-20T09:57:06 |
| description | |
| device_id | f83ade73-16a7-49f6-b52f-3a2b7e4a638e |
| device_owner | compute:nova |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "81521c0d-232e-411b-9481-a2fd2438f64c", "ip_address": "172.16.1.3"} |
| id | 40e7b1f7-1127-408f-9460-fbfb3f0acc74 |
| mac_address | fa:16:3e:23:97:8c |
| name | |
| network_id | 5fce6697-649b-4708-9d3c-ec882b9244cb |
| security_groups | 151930cd-7cd3-4581-9714-d660cbcb2de3 |
| status | ACTIVE |
| tenant_id | fc74c6ec4f3f48eb9a9c6526c36b297a |
| updated_at | 2017-02-20T09:57:50 |
+-----------------------+-----------------------------------------------------------------------------------+
- Iptables
原理很简单，neutron通过iptables把vip的源mac地址换成实际虚拟机port的mac地址。
:neutron-openvswi-s40e7b1f7-1 - [0:0]
-A neutron-openvswi-o40e7b1f7-1 -j neutron-openvswi-s40e7b1f7-1
-A neutron-openvswi-s40e7b1f7-1 -s 172.16.1.40/32 -m mac --mac-source FA:16:3E:23:97:8C -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-s40e7b1f7-1 -s 172.16.1.3/32 -m mac --mac-source FA:16:3E:23:97:8C -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-s40e7b1f7-1 -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP
[root@mitaka ~(keystone_admin)]# iptables-save |grep neutron-openvswi-sf7364078-7
:neutron-openvswi-sf7364078-7 - [0:0]
-A neutron-openvswi-of7364078-7 -j neutron-openvswi-sf7364078-7
-A neutron-openvswi-sf7364078-7 -s 172.16.1.40/32 -m mac --mac-source FA:16:3E:04:7A:5B -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-sf7364078-7 -s 172.16.1.2/32 -m mac --mac-source FA:16:3E:04:7A:5B -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-sf7364078-7 -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP
