REDIS高可用集群的搭建和运维
2020-03-07 本文已影响0人
chuan_bai
一、REDIS架构图
redis集群是一个由多个主从节点群组成的分布式服务器群，它具有复制、高可用和分片特性。
REDIS Cluster架构图
二、安装REDIS
- 从官网下载Redis的最新稳定版，本次操作使用的是5.0.7版本
wget http://download.redis.io/releases/redis-5.0.7.tar.gz
- 把压缩包放在/opt/目录下，解压压缩包，在opt下会生成redis-5.0.7目录
tar -xvf redis-5.0.7.tar.gz
- 进入/opt/redis-5.0.7/src目录编译并安装redis
cd /redis-5.0.7/src
make & make install
- 如在安装过程中出现缺少tcl8.5的错误，运行以下命令安装tcl8.5
sudo apt-get install tcl8.5
- 指定配置文件启动redis（如果需要后台启动，则需要修改redis.conf中的daemonize
redis-server ../redis.conf
三、搭建REDIS集群
下面开始redis的集群搭建（由于只有一台服务器，所以在一台服务器上搭建伪集群，多台服务器同一台服务器搭建方法一样
- 在opt下面新建rediscluster文件夹
mkdir /opt/rediscluster
- 在其中新建8000、8001、8002、8003、8004、8005 六个文件，分别对应到时候配置的redis端口
mkdir 8000 、mkdir 8001 、mkdir 8002
mkdir 8003 、mkdir 8004 、mkdir 8005
- 把之前的redis.conf 文件复制到8000下
cp /opt/redis-5.0.7/redis.conf /opt/rediscluser/8000
- 打开redis.conf 修改如下内容
#后台启动
daemonize yes
#端口号
port 8000
#指定数据存放位置，每个节点对应一个位置
dir /opt/rediscluster/8000/
#启动集群模式
cluster-enabled yes
#集群节点信息文件
cluster-config-file nodes-8000.conf
#集群超时事件
cluster-node-timeout 5000
#去掉bind绑定访问ip信息，让外网可以访问
# bind 127.0.0.1
#关闭保护模式
protected-mode no
#开启AOF持久化
appendonly yes
#设置redis访问密码
requirepass 123456
#设置集群节点间访问密码
masterauth 123456
其他几个redis实例也做相同的操作分别复制到8001-8005几个文件夹下，配置对应的redis端口
- 分别启动六个redis实例
/opt/redis-5.0.7/src/redis-server /opt/rediscluster/8000/redis.conf
会出现以下信息
root@iZbp15jhfodzncs996bwpdZ:/opt/redis-5.0.7# /opt/redis-5.0.7/src/redis-server /opt/rediscluster/8000/redis.conf
9369:C 08 Mar 2020 14:39:09.157 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9369:C 08 Mar 2020 14:39:09.157 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=9369, just started
9369:C 08 Mar 2020 14:39:09.157 # Configuration loaded
- ps -ef | grep redis 查看是否启动成功
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8001# ps -ef|grep redis
root 9579 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8000 [cluster]
root 9584 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8001 [cluster]
root 9589 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8002 [cluster]
root 9594 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8003 [cluster]
root 9599 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8004 [cluster]
root 9604 1 0 14:47 ? 00:00:00 /opt/redis-5.0.7/src/redis-server *:8005 [cluster]
root 9609 9220 0 14:47 pts/0 00:00:00 grep --color=auto redis
- 创建Redis集群
redis-cli -a 123456 --cluster create --cluster-replicas 1 127.0.0.1:8000 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005
这个1代表一个主节点一个从节点，组成一个cluster集群至少需要3个主节点，所以至少需要启动6个Redis实例
- 下图代表集群已经启动成功
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8001# redis-cli -a 123456 --cluster create --cluster-replicas 1 127.0.0.1:8000 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:8004 to 127.0.0.1:8000
Adding replica 127.0.0.1:8005 to 127.0.0.1:8001
Adding replica 127.0.0.1:8003 to 127.0.0.1:8002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[0-5460] (5461 slots) master
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5461-10922] (5462 slots) master
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[10923-16383] (5461 slots) master
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 127.0.0.1:8000)
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
slots: (0 slots) slave
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
slots: (0 slots) slave
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
slots: (0 slots) slave
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
9.进入redis
redis-cli -a 123456 -c -h 127.0.0.1 -p 8000
- 用以下命令进行验证,先进入8000端口的redis存入name，在进入8001查看name看是否能查到
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8001# redis-cli -a 123456 -c -h 127.0.0.1 -p 8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:8000> set name chuan.bai
-> Redirected to slot [5798] located at 127.0.0.1:8001
OK
127.0.0.1:8001> exit
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8001# redis-cli -a 123456 -c -h 127.0.0.1 -p 8001
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:8001> get name
"chuan.bai"
- 其他重要命令
#查看集群信息
cluster info
#查看节点信息
cluster nodes
#关闭集群
/opt/redis-5.0.7/src/redis-cli -a 123456 -c -h 127.0.0.1 -p 8000 shutdown
四、新增主节点，并迁移槽位
- 再启动两个redis实例分别为8006、8007端口
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# /opt/redis-5.0.7/src/redis-server /opt/rediscluster/8006/redis.conf
9684:C 08 Mar 2020 15:00:51.963 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9684:C 08 Mar 2020 15:00:51.963 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=9684, just started
9684:C 08 Mar 2020 15:00:51.963 # Configuration loaded
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# /opt/redis-5.0.7/src/redis-server /opt/rediscluster/8007/redis.conf
9689:C 08 Mar 2020 15:00:55.929 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9689:C 08 Mar 2020 15:00:55.929 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=9689, just started
9689:C 08 Mar 2020 15:00:55.929 # Configuration loaded
- 把8006端口的redis实例作为主节点加入到已存在的节点中
redis-cli -a 123456 --cluster add-node 127.0.0.1:8006 127.0.0.1:8000
- 出现以下信息代表成功了
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster add-node 127.0.0.1:8006 127.0.0.1:8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 127.0.0.1:8006 to cluster 127.0.0.1:8000
>>> Performing Cluster Check (using node 127.0.0.1:8000)
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
slots: (0 slots) slave
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
slots: (0 slots) slave
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
slots: (0 slots) slave
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:8006 to make it join the cluster.
[OK] New node added correctly.
- 此时用cluster nodes命令可以看到8006作为主节点已加入集群，但是还未分配槽位,我们需要为新节点手工分配hash槽
127.0.0.1:8000> cluster nodes
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583651427000 3 connected 10923-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583651427713 4 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583651429000 1 connected 0-5460
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583651428716 2 connected 5461-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583651429719 5 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583651428000 6 connected
32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006@18006 master - 0 1583651428000 0 connected
- 使用reshard命令进行迁移分配
redis-cli -a 123456 --cluster reshard 127.0.0.1:8000
会出现以下信息，选择需要迁移多少个分片到新的节点
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster reshard 127.0.0.1:8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 127.0.0.1:8000)
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
slots: (0 slots) slave
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
slots: (0 slots) slave
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
slots: (0 slots) slave
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
M: 32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006
slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 选择迁移800个分片到新的节点
How many slots do you want to move (from 1 to 16384)? 800 #输入片区数
What is the receiving node ID?
跳出来需要选择指定的节点分片的提示，接下来选择节点
- 选择实例8006节点
#键入8006节点编号
What is the receiving node ID? 32a83a0f20017e018e522215f616c16e6ff7c96e
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs
- 根据提示键入all选择800个分片从其他节点平均分出来,如果不平均分也可以从指定节点分
#选择平均分
Source node #1: all
Ready to move 800 slots.
Source nodes:
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
Destination node:
M: 32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006
slots: (0 slots) master
#分片计划
Resharding plan:
Moving slot 5461 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5462 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5463 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5464 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5465 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5466 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5467 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5468 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5469 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5470 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5471 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
Moving slot 5472 from 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
........
- 键入yes确认分片,集群即开始分片，这过程中如果redis正在运行可能会有卡顿的情况
#是否开始执行分片
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 5461 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5462 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5463 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5464 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5465 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5466 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5467 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5468 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5469 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5470 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5471 from 127.0.0.1:8001 to 127.0.0.1:8006:
Moving slot 5472 from 127.0.0.1:8001 to 127.0.0.1:8006:
- 使用cluster nodes命令可以看到8006节点已分配槽位
#查看节点情况，是否已为8006节点分配槽位
127.0.0.1:8000> cluster nodes
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583652828000 3 connected 11189-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583652825000 4 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583652824000 1 connected 266-5460
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583652827102 2 connected 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583652829108 5 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583652827000 6 connected
32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006@18006 master - 0 1583652828105 7 connected 0-265 5461-5727 10923-11188
五、新增从节点
- 添加8007节点到集群中
redis-cli -a 123456 --cluster add-node 127.0.0.1:8006 127.0.0.1:8000
oot@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster add-node 127.0.0.1:8007 127.0.0.1:8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 127.0.0.1:8007 to cluster 127.0.0.1:8000
>>> Performing Cluster Check (using node 127.0.0.1:8000)
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[266-5460] (5195 slots) master
1 additional replica(s)
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[11189-16383] (5195 slots) master
1 additional replica(s)
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
slots: (0 slots) slave
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5728-10922] (5195 slots) master
1 additional replica(s)
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
slots: (0 slots) slave
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
slots: (0 slots) slave
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
M: 32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006
slots:[0-265],[5461-5727],[10923-11188] (799 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:8007 to make it join the cluster.
[OK] New node added correctly
- 进入到8007的客户端
redis-cli -a 123456 -c -h 127.0.0.1 -p 8007
- 加入指定主节点8007作为从节点
127.0.0.1:8007> cluster replicate 32a83a0f20017e018e522215f616c16e6ff7c96e
显示ok代表成功
- 使用命令cluster nodes 发现从节点已注册成功
127.0.0.1:8007> cluster nodes
32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006@18006 master - 0 1583653693000 7 connected 0-265 5461-5727 10923-11188
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583653696769 3 connected 11189-16383
ce8499789a8f4c6e69cccb81063a9834fbf9a5a8 127.0.0.1:8007@18007 myself,slave 32a83a0f20017e018e522215f616c16e6ff7c96e 0 1583653695000 0 connected
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583653696000 2 connected 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583653694000 2 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 master - 0 1583653695000 1 connected 266-5460
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583653694000 1 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583653697772 3 connected
六、删除节点
- 回收分片
redis-cli -a 123456 --cluster reshard 127.0.0.1:8000
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster reshard 127.0.0.1:8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 127.0.0.1:8000)
M: dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000
slots:[266-5460] (5195 slots) master
1 additional replica(s)
S: ce8499789a8f4c6e69cccb81063a9834fbf9a5a8 127.0.0.1:8007
slots: (0 slots) slave
replicates 32a83a0f20017e018e522215f616c16e6ff7c96e
M: 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002
slots:[11189-16383] (5195 slots) master
1 additional replica(s)
S: 4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003
slots: (0 slots) slave
replicates dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
M: 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001
slots:[5728-10922] (5195 slots) master
1 additional replica(s)
S: 91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004
slots: (0 slots) slave
replicates 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e
S: 7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005
slots: (0 slots) slave
replicates 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b
M: 32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006
slots:[0-265],[5461-5727],[10923-11188] (799 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 把8006实例中的800个槽位迁移到8000实例中
How many slots do you want to move (from 1 to 16384)? 800 #800个槽位
What is the receiving node ID? dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346
Please enter all the source node IDs. #8000实例id
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: 32a83a0f20017e018e522215f616c16e6ff7c96e #8006实例id
Source node #2: done #键入done
3.cluster nodes 命令可以查看到8006的槽位已经全部移到了8000节点中了
127.0.0.1:8000> cluster nodes
ce8499789a8f4c6e69cccb81063a9834fbf9a5a8 127.0.0.1:8007@18007 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583654865000 8 connected
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583654867000 3 connected 11189-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583654868020 8 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583654865000 8 connected 0-5727 10923-11188
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583654868521 2 connected 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583654866000 5 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583654864011 6 connected
32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006@18006 master - 0 1583654867018 7 connected
- 删除从8007节点
redis-cli -a 123456 --cluster del-node 127.0.0.1:8007
出现以下信息代表8007节点已删除
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster del-node 127.0.0.1:8007 ce8499789a8f4c6e69cccb81063a9834fbf9a5a8
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node ce8499789a8f4c6e69cccb81063a9834fbf9a5a8 from cluster 127.0.0.1:8007
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
- cluster nodes命令可以查看到节点已经删除了
127.0.0.1:8000> cluster nodes
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583655124000 3 connected 11189-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583655124000 8 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583655122000 8 connected 0-5727 10923-11188
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583655124000 2 connected 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583655122000 5 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583655123000 6 connected
32a83a0f20017e018e522215f616c16e6ff7c96e 127.0.0.1:8006@18006 master - 0 1583655124622 7 connected
- 删除主节点
redis-cli -a 123456 --cluster del-node 127.0.0.1:8000 32a83a0f20017e018e522215f616c16e6ff7c96e #8006节点的id
8007节点已关闭
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 --cluster del-node 127.0.0.1:8000 32a83a0f20017e018e522215f616c16e6ff7c96e
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 32a83a0f20017e018e522215f616c16e6ff7c96e from cluster 127.0.0.1:8000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
可以看到8007节点已移除
127.0.0.1:8000> cluster nodes
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583655443000 3 connected 11189-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583655443377 8 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583655440000 8 connected 0-5727 10923-11188
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583655445381 2 connected 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583655444000 5 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583655444379 6 connected
七、均衡分片命令
使用这个命令时redis可能会有停顿
- 输入命令
redis-cli -a 123456 -c --cluster rebalance 127.0.0.1:8000
2.出现以下界面
root@iZbp15jhfodzncs996bwpdZ:/opt/rediscluster/8007# redis-cli -a 123456 -c --cluster rebalance 127.0.0.1:8000
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 127.0.0.1:8000)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Rebalancing across 3 nodes. Total weight = 3.00
Moving 267 slots from 127.0.0.1:8000 to 127.0.0.1:8002
###########################################################################################################################################################################################################################################################################
Moving 266 slots from 127.0.0.1:8000 to 127.0.0.1:8001
##########################################################################################################################################################################################################################################################################
- 可以查看到基本上槽位平均分配了
127.0.0.1:8000> cluster nodes
3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 127.0.0.1:8002@18002 master - 0 1583656076000 9 connected 0-266 11189-16383
4f7a83dd69028ba30d2ba0494a680f3720fda72f 127.0.0.1:8003@18003 slave dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 0 1583656077000 8 connected
dd0f8096ef6fa34fc46c3bc6089aa8e6909a1346 127.0.0.1:8000@18000 myself,master - 0 1583656073000 8 connected 533-5727 10923-11188
232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 127.0.0.1:8001@18001 master - 0 1583656075000 10 connected 267-532 5728-10922
91fdf9f6c4d8773bcd3810bc7136f955c2f7291b 127.0.0.1:8004@18004 slave 232aad45e63f11fda0fc1971c00dc9dfe8e2db8e 0 1583656077971 10 connected
7330d6fe71c37a2e52a221b1beea221b8ecea47f 127.0.0.1:8005@18005 slave 3d1f1d54a9a225e56ae7730f0fbd0fed4a97a85b 0 1583656074000 9 connected
八、原理分析
Redis Cluster 会将所有数据分成16384的slots（槽位），每个节点负责其中一部分槽位，槽位的信息存储在每个节点中。
槽位定位算法：
Cluster默认会对key值进行使用crc16算法进行hash得到一个整数值，然后用这个整数值堆16384进行取模来得到具体槽位
HASH_SLOT=ORC16(key)mode 16384
