Redis高可用集群搭建最佳实践
文件准备
redis-3.0.6.tar.gz
redis-3.3.0.gem
ruby-2.3.0.tar.gz
zlib-1.2.11.tar.gz
安装redis
前置工作
在~/soft/下解压文件
tar -zxvf redis-3.0.6.tar.gz
移动到~/
mv redis-3.0.6 ../
安装
进入redis-3.0.6 目录,进行安装 PREFIX指定安装目录
make PREFIX=~/redis-3.0.6 install
安装成功后，~/redis-3.0.6/src/下会生成许多脚本，包括 redis-cli redis-server redis-trib.rb
创建集群目录
在 ~/ 目录下新增集群目录
mkdir redis_cluster
在redis_cluster下新增bin目录，并复制~/redis-3.0.6/src下的如下脚本到该目录下，方便运维
cp mkreleasehdr.sh redis-benchmark redis-cli redis-server redis-trib.rb ~/redis_cluster/bin
在redis_cluster 新增6个节点目录 7101-7106
mkdir 7101 7102 7103 7104 7105 7106
在7101-7106目录下新增data 、 logs 、 conf三个目录
复制 ~/redis-3.0.6/conf/redis.conf 到7101-7106的conf目录下
cp ~/redis-3.0.6/conf/redis.conf ~/redis_cluster/7101/conf ~/redis_cluster/7102/conf ~/redis_cluster/7103/conf ~/redis_cluster/7104/conf ~/redis_cluster/7105/conf ~/redis_cluster/7106/conf
编辑7101/conf下的redis.conf
port 7101
#本主机地址
bind 主机ip
#日志文件
logfile "~/redis_cluster/7101/log/redis-log-7101.log"
#备份数据文件
dbfilename dump-7101.rdb
#备份数据文件生成目录
dir ~/redis_cluster/7101/data
#设置为集群模式
cluster-enabled yes
#设置为后台进程
daemonize yes
masterauth passwd123
requirepass passwd123
依次修改7102-7106/conf/下的redis.conf 只需修改如下信息
#redis节点端口
port 7102
#本主机地址
bind 主机ip
#日志文件
logfile "~/redis_cluster/7101/log/redis-log-7102.log"
#备份数据文件
dbfilename dump-7102.rdb
#备份数据文件生成目录
dir ~/redis_cluster/7102/data
为了方便维护，在~/redis_cluster/bin下新增 start_all.sh 启动所有redis服务
vim start_all.sh
./redis-server ../7101/conf/redis.conf
./redis-server ../7102/conf/redis.conf
./redis-server ../7103/conf/redis.conf
./redis-server ../7104/conf/redis.conf
./redis-server ../7105/conf/redis.conf
./redis-server ../7106/conf/redis.conf
为了方便维护，在~/redis_cluster/bin下新增 stop_all.sh 启动所有redis服务
vim stop_all.sh
# -h x.x.x.x 代表主机 -p 代表端口 -a 代表密码
./redis-cli -h x.x.x.x -p 7101 -a 123456 shutdown
./redis-cli -h x.x.x.x -p 7102 -a 123456 shutdown
./redis-cli -h x.x.x.x -p 7103 -a 123456 shutdown
./redis-cli -h x.x.x.x -p 7104 -a 123456 shutdown
./redis-cli -h x.x.x.x -p 7105 -a 123456 shutdown
./redis-cli -h x.x.x.x -p 7106 -a 123456 shutdown
安装ruby
redis集群启动需要依赖ruby，如果系统没有默认安装ruby,则需手动安装
前置工作
解压
tar –zxvf ruby-2.3.0.tar.gz
移动到 ~/目录
mv ruby-2.3.0 ~/
安装
进入~/ruby-2.3.0 安装–prefix 指定安装目录
./configure --prefix=/data/airedis/ruby
make
make install
ruby加入环境变量
image.png
安装redis-3.3.0.gem
安装安装集权启动脚本依赖redis-3.3.6.gem，类似于java中的jar
进入ruby安装目录 cd ~/ruby/bin
./gem install --local ~/soft/redis-3.3.0.gem
如果报错 ,则需安装zlib
ERROR: Loading command: install (LoadError)
cannot load such file -- zlib
安装zlib
前置工作
解压
tar -zxvf zlib-1.2.11.tar.gz
移动到 ~/目录
mv zlib-1.2.11 ~/
安装
进入~/zlib-1.2.11目录， –prefix指定安装路径
./configure --prefix=~/ruby/zlib
make
make install
ruby添加zlib依赖
给ruby添加zlib依赖，进入~/ruby/bin目录，执行如下命令 ，其中-with-zlib-dir 指定zlib安装目录
~/ruby/bin/ruby extconf.rb --with-zlib-dir=~/ruby/zlib
make
make install
启动redis集群
先启动所有redis节点服务
进入~/redis_cluster/bin 目录,执行start_all.sh
./start_all.sh
创建集群
进入~/redis_cluster/bin目录
./redis-trib.rb create --replicas 1 ip地址:7101 ip地址:7102 ip地址:7103 ip地址:7104 ip地址:7105 ip地址:7106
如果报错
redis [ERR] Sorry, can't connect to node ip地址:7101
是因为redis设置了密码，单机连接都OK，但是创建集群就报错Sorry, can’t connect to node
需要修改~/ruby/lib/ruby/gems/2.3.0/gems/redis-3.3.0/lib/redis/client.rb 文件
可能大家路径不一样 则查找 一下 find ~ -name client.rb
修改 vim client.rb 的password
DEFAULTS = {
:url => lambda { ENV["REDIS_URL"] },
:scheme => "redis",
:host => "127.0.0.1",
:port => 6379,
:path => nil,
:timeout => 5.0,
:password => "xxxxxx",
:db => 0,
:driver => nil,
:id => nil,
:tcp_keepalive => 0,
:reconnect_attempts => 1,
:inherit_socket => false
集群创建成功信息如下：
Creating cluster
Performing hash slots allocation on 6 nodes...
Using 3 masters:
x.x.x.x:7101
x.x.x.x:7102
x.x.x.x:7103
Adding replica x.x.x.x:7104 to x.x.x.x:7101
Adding replica x.x.x.x:7105 to x.x.x.x:7102
Adding replica x.x.x.x:7106 to x.x.x.x:7103
M: e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3 x.x.x.x:7101
slots:0-5460 (5461 slots) master
M: a70222aaec8bf3544b9f132832d8c3e532aac0bc x.x.x.x:7102
slots:5461-10922 (5462 slots) master
M: 2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4 x.x.x.x:7103
slots:10923-16383 (5461 slots) master
S: ec24032f700f18902fea6047c61a08900648a36d x.x.x.x:7104
replicates e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3
S: 62c8478c5ec26fe588c0bf5a882ef99f6687717f x.x.x.x:7105
replicates a70222aaec8bf3544b9f132832d8c3e532aac0bc
S: ec2cc8f1c9eee20f1231e2885b384d95689948d9 x.x.x.x:7106
replicates 2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4
Can I set the above configuration? (type 'yes' to accept): yes
Nodes configuration updated
Assign a different config epoch to each node
Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
Performing Cluster Check (using node x.x.x.x:7101)
M: e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3 x.x.x.x:7101
slots:0-5460 (5461 slots) master
M: a70222aaec8bf3544b9f132832d8c3e532aac0bc x.x.x.x:7102
slots:5461-10922 (5462 slots) master
M: 2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4 x.x.x.x:7103
slots:10923-16383 (5461 slots) master
M: ec24032f700f18902fea6047c61a08900648a36d x.x.x.x:7104
slots: (0 slots) master
replicates e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3
M: 62c8478c5ec26fe588c0bf5a882ef99f6687717f x.x.x.x:7105
slots: (0 slots) master
replicates a70222aaec8bf3544b9f132832d8c3e532aac0bc
M: ec2cc8f1c9eee20f1231e2885b384d95689948d9 x.x.x.x:7106
slots: (0 slots) master
replicates 2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4
[OK] All nodes agree about slots configuration.
Check for open slots...
Check slots coverage...
[OK] All 16384 slots covered.
查看集群信息
~/redis_cluster/bin$./redis-cli -h x.x.x.x -p 7101 -c -a 123456 ,其中 -a 123456 代表设置的密码
ip地址:7101> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_sent:590
cluster_stats_messages_received:590
查看具体集群节点信息
x.x.x.x:7101> cluster nodes
ec2cc8f1c9eee20f1231e2885b384d95689948d9 x.x.x.x:7106 slave 2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4 0 1569570540713 6 connected
a70222aaec8bf3544b9f132832d8c3e532aac0bc x.x.x.x:7102 master - 0 1569570546721 2 connected 5461-10922
ec24032f700f18902fea6047c61a08900648a36d x.x.x.x:7104 slave e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3 0 1569570545719 4 connected
2e03fbf49b2a7e1c4ca5f8499aa1f5c6488121d4 x.x.x.x:7103 master - 0 1569570543716 3 connected 10923-16383
e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3 x.x.x.x:7101 myself,master - 0 0 1 connected 0-5460
62c8478c5ec26fe588c0bf5a882ef99f6687717f x.x.x.x:7105 slave a70222aaec8bf3544b9f132832d8c3e532aac0bc 0 1569570544717 5 connected
调整主从节点复制从属关系
随机分配主从节点复制，如果是同台主机，需要调整到不同的主机
调整方法为：7006更改隶属关系时，直接登录指定要更改隶属关系的IP、端口
-a 123456 表示密码 -c表示集群登录
./redis-cli -c -h ip地址 -p 7006 -a 123456
直接指定新的master节点的ID
cluster replicate e54b115b0d2a066544cdf1c436b0e45e5e9eaeb3
1569576145101
上图原本7106从节点从7103主节点复制数据
修改为从7101主节点复制数据
