RabbitMQ部署在DockerSwarm集群

在现在的docker swarm 环境上搭建RabbitMQ（文中简称mq），

IP	角色
172.16.10.172	master
172.16.10.173	node1
172.16.10.174	node2

使用docker-compose编排构建集群

参考 oprearocks/RabbitMQ-Docker-cluster进行forked后修改为适合自己的，看源码的文件列表

按作者的源码，需要将 definitions.json 和 rabbitmq.config 放到所有需要mq的机器上，看源码的docker-compose.yml 其中需要将两个配置文件映射到容器中

volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"

并且还需要事先创建network

docker network create rabbitmq-cluster

以下是我修改后的版本 修改后的源码

修改后的docker-compose.yml，不需要在每台mq的机器上放置两个配置文件，而是使用了docker-compose编排3.3版本的docker config

version: "3.3"
services:
  rabbit1:
    image: rabbitmq:3-management
    hostname: rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit1
    depends_on:
      - rabbit3
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"

    configs:
      - source: rabbitmq_config
        target: /etc/rabbitmq/rabbitmq.config
      - source: definitons_json
        target: /etc/rabbitmq/definitions.json

    networks:
      - rabbitmq-cluster

  rabbit2:
    image: rabbitmq:3-management
    hostname: rabbit2

    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit2
    depends_on:
      - rabbit3
    configs:
      - source: rabbitmq_config
        target: /etc/rabbitmq/rabbitmq.config
      - source: definitons_json
        target: /etc/rabbitmq/definitions.json

    networks:
      - rabbitmq-cluster
    
  rabbit3:
    image: rabbitmq:3-management
    hostname: rabbit3
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit3

    configs:
      - source: rabbitmq_config
        target: /etc/rabbitmq/rabbitmq.config
      - source: definitons_json
        target: /etc/rabbitmq/definitions.json

    networks:
      - rabbitmq-cluster

configs:
  rabbitmq_config:
    file: ./rabbitmq.config
  definitons_json:
    file: ./definitions.json

networks:
  rabbitmq-cluster:
    driver: overlay

只需要在 dockerswarm的manager角色的节点上下载代码

git clone https://github.com/liangxiaobo/RabbitMQ-Docker-cluster.git

进入目录执行部署

[root@master RabbitMQ-Docker-cluster]# docker stack deploy -c docker-compose.yml rabbit
Creating network rabbit_rabbitmq-cluster
Creating config rabbit_rabbitmq_config
Creating config rabbit_definitons_json
Creating service rabbit_rabbit3
Creating service rabbit_rabbit1
Creating service rabbit_rabbit2

查看服务

root@master RabbitMQ-Docker-cluster]# docker stack services rabbit 
ID                  NAME                MODE                REPLICAS            IMAGE                   PORTS
2e59unar85ko        rabbit_rabbit3      replicated          1/1                 rabbitmq:3-management   
lec34upa7hs2        rabbit_rabbit2      replicated          1/1                 rabbitmq:3-management   
otizbi69uixu        rabbit_rabbit1      replicated          1/1                 rabbitmq:3-management   *:4369->4369/tcp, *:5671-5672->5671-5672/tcp, *:15671-15672->15671-15672/tcp, *:25672->25672/tcp

成功后访问 http://172.16.10.172:15672 默认用户名和密码都是guest

能看到集群的三个mq节点

看 rabbitmq.config

[
    { rabbit, [
        {cluster_nodes, {[ 'rabbit3@rabbit3' ], disc}},
    {tcp_listen_options, [binary, {backlog, 1024}, {nodelay, true}, {keepalive, true} ]},
    {vm_memory_high_watermark, 0.6},
        { loopback_users, [ ] },
        { tcp_listeners, [ 5672 ] },
        { ssl_listeners, [ ] },
        { default_pass, <<"password">> },
        { default_user, <<"admin">> },
        { hipe_compile, false }
    ] },

    { rabbitmq_management, [ { listener, [
        { port, 15672 },
        { ssl, false }
    ] },
        {load_definitions, "/etc/rabbitmq/definitions.json"}
    ] }
].

注意看 cluster_nodes

{cluster_nodes, {[ 'rabbit3@rabbit3' ], disc}},

cluster_nodes
当节点第一次启动的时候，设置此选项会导致集群动作自动发生. 元组的第一个元素是其它节点想与其建立集群的节点. 第二个元素是节点的类型，要么是disc,要么是ram

有可能你创建的集群只能看到两个节点，那可能是因为docker swarm创建服务的顺序问题造成的，如果哪个节点没有加入集成，可以手动加入到集群中

WX20181126-180635@2x.png

进入容器后执行

rabbit@master:~$ docker container exec -it [容器ID] /bin/bash
root@rabbit3:/# rabbitmqctl stop_app
Stopping rabbit application on node rabbit3@rabbit3 ...
root@rabbit3:/# rabbitmqctl join_cluster rabbit1@rabbit1
Clustering node rabbit3@rabbit3 with rabbit1@rabbit1
root@rabbit3:/# rabbitmqctl start_app
Starting node rabbit3@rabbit3 ...
 completed with 3 plugins.

修改后的源码
rabbitmq.config详细配置参数
Clustering Guide