十一、RocketMQ 其他内容
2019-04-15 本文已影响3人
ASD_92f7
一、概述
参考链接:
http://rocketmq.apache.org/docs/rmq-arc/
http://rocketmq.apache.org/docs/rmq-deployment/
http://rocketmq.apache.org/docs/cli-admin-tool/
二、RocketMQ Architecture 结构
RocketMQ工作结构图RocketMQ主要由4部分组成:
-
name servers
服务发现及路由 -
brokers
broker模块
- Remoting Module:远程模块,负责处理客户端的请求(producer、consumers、cmd line)
- Client Manager:管理Producers/Consumers,并且维护Consumers的Topic订阅
- Store Service:支持用简单的API对硬盘上的消息进行查询、存储
- HA Service:提供Master与Slaver之间的数据同步
- Index Service:根据给定的key简历消息索引
- producers
- consumers
三、Deployment(部署)
-
Name Server部署建议
建议部署2个以上的Name Server -
Broker
A broker set contains one master with brokerId set to 0 and several slaves with non-zero brokerIDs
一个broker集合包括一个bokerId = 0的master,以及一些 brokerIDs !=0 的slave,在一个集合中的broker必须拥有相同的brokerName,
- Configuration 建议的配置如下:
Property Name | 默认值 | 说明 |
---|---|---|
listenPort | 10911 | broker端口 |
namesrvAddr | null | name server 地址 |
brokerIP1 | IP地址 | 如果有多个网络地址则必须指定 |
brokerName | null | broker name |
brokerClusterName | DefaultCluster | 指明当前broker属于哪个集群 |
brokerId | 0 | broker id,0为master, 非0整数为slave |
storePathCommitLog | $HOME/store/commitlog/ | commit log地址 |
storePathConsumerQueue | $HOME/store/consumequeue/ | 消费者队列文件地址 |
mapedFileSizeCommitLog | 1024 * 1024 * 1024(1G) | 日志映射文件的大小 |
deleteWhen | 4 | 超过多长时间删除日志,这个单位待确认 |
fileReserverdTime | 72 | 日志保留时间,单位hour,与上面啥区别 |
brokerRole | ASYNC_MASTER | SYNC_MASTER/ASYNC_MASTER/SLVAE |
flushDiskType | ASYNC_FLUSH | {SYNC_FLUSH/ASYNC_FLUSH}. SYNC_FLUSH模式,broker在反馈producer之前要完成flush. ASYNC_FLUSH模式效率更高 |
- CLI Admin Tool 命令行工具
常用命令行命令如下:
updateTopic Update or create topic
deleteTopic Delete topic from broker and NameServer
updateSubGroup Update or create subscription group
deleteSubGroup Delete subscription group from broker
updateBrokerConfig Update broker's config
updateTopicPerm Update topic perm
topicRoute Examine topic route info
topicStatus Examine topic Status info
topicClusterList get cluster info for topic
brokerStatus Fetch broker runtime status data
queryMsgById Query Message by Id
queryMsgByKey Query Message by Key
queryMsgByUniqueKey Query Message by Unique key
queryMsgByOffset Query Message by offset
queryMsgByUniqueKey Query Message by Unique key
printMsg Print Message Detail
sendMsgStatus Send msg to broker
brokerConsumeStats Fetch broker consume stats data
producerConnection Query producer's socket connection and client version
consumerConnection Query consumer's socket connection, client version and subscription
consumerProgress Query consumers's progress, speed
consumerStatus Query consumer's internal data structure
cloneGroupOffset Clone offset from other group
clusterList List all of clusters
topicList Fetch all topic list from name server
updateKvConfig Create or update KV config
deleteKvConfig Delete KV config
wipeWritePerm Wipe write perm of broker in all name server
resetOffsetByTime Reset consumer offset by timestamp(without client restart)
updateOrderConf Create or update or delete order conf
cleanExpiredCQ Clean expired ConsumeQueue on broker.
cleanUnusedTopic Clean unused topic on broker
startMonitoring Start Monitoring
statsAll Topic and Consumer tps stats
syncDocs Synchronize wiki and issue to github.com
allocateMQ Allocate MQ
checkMsgSendRT Check message send response time
clusterRT List All clusters Message Send RT
- pre-built config(预设配置)
在 ~/rocketmq/conf下有几个预设的配置文件,方便启动部署,注意,这些配置中的flush方式都是:ASYNC_FLUSH
- 2m-2s-sync:多个master,多个slave(SYNC_MASTER)
- 2m-2s-async:多个master,多个slave(ASYNC_MASTER)
- 2m-noslave:多个master,无slave模式
以 2m-2s-sync 为例,启动方式如下:
>cd /home/rocketmq/dist/bin
>bash mqbroker -c ../conf/2m-2s-sync/broker-a.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-a-s.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b-s.properties -n 192.168.0.2:9876,192.168.0.3:9876
How to verify
Execute the following command to verify according to the CLI section:
> bash mqadmin clusterlist
上面的例子,启动了2对broker(master-slave)