Codis 实现 Redis 集群（上）
2020-08-05 本文已影响0人
Alexander_Zz
Codis 是一个分布式 Redis 解决方案，对上层的应用来说，连接到 Codis Proxy 和连接原生的 Redis Server 没有明显的区别（不支持的命令列表），上层应用可像使用单机的 Redis 一样使用，Codis 底层会处理请求的转发，不停机的数据迁移等工作，所有后边的一切事情，对于前面的客户端来说是透明的，可简单的认为后边连接的是一个内存无限大的 Redis 服务
Codis 是豌豆荚的开源方案，目前在 Redis 集群实现方式对比，Codis 集群比较稳定的方案，并且客户端不需要做任何修改，相对 Redis Cluster 兼容性更强，可节约大量开发成本并减少大量后期维护成本，豌豆荚 GitLab，Codis GitLab
Codis 特点
- 可无缝迁移到 Codis，自带迁移工具，且案例较多
- 可动态扩容和缩容
- 多业务完全透明，业务不知道运行的是 Codis
- 支持多核心 CPU，twemproxy 只能单核
- Codis 是 i 有中心基于 proxy 的设计，是客户端像连接单机一样操作 proxy
- 有部分命令不能支持，比如
keys *等 - 支持 group 划分，组内可设置一主多从，通过 sentinel 监控 Redis 主从，当 Master 挂掉后自动将 Slave 切换为 Master
- 设置的进程要最大等于 CPU 的核心，不能超过 CPU 的核心数
- 其依赖于 zookeeper，里面保存的是 key 保存的 Redis 主机位置，因此 zookeeper 要做高可用
- 监控可使用接口和 dashboard
- tidb 是豌豆荚团队实现的分布式 MySQL 数据库，GitLab 地址
安装 Go 环境，Codis 基于 Go 开发
架构环境
Codis proxy 相当于 Redis，即连接 Codis proxy 和连接 Redis 是没有任何区别的，Codis proxy 无状态，不负责记录是否在哪保存，数据在 zookeeper 记录，即 Codis proxy 向 zookeeper 查询 key 的记录位置，proxy 将请求转发到一个组进行处理，一个组里面有一个 Master 和一个或多个 Slave 组成，默认有 1024 个槽位，Redis Cluster 默认有 16384 个槽位，其把不同的槽位的内容放在不通的 group
-
部署环境
架构图
Codis 是基于 Go 语言编写的，因此要安装 Go 语言环境
shell> mkdir /opt/gopath
shell> vim /etc/profile
export GOPATH=/opt/gopath
shell> . /etc/profile
shell> echo $GOPATH
/opt/gopath
每台服务器安装 java 环境和 zookeeper，zookeeper 集群最少需要 3 台服务器，推荐 5 台，因为 zookeeper 是 java 的
shell> yum install java
shell> cd /usr/local/src/
shell> wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
shell> tar zxvf zookeeper-3.4.6.tar.gz
shell> ln -sv /usr/local/src/zookeeper-3.4.6 /usr/local/zookeeper # 创建软连接
‘/usr/local/zookeeper’ -> ‘/usr/local/src/zookeeper-3.4.6’
shell> cd /opt/
shell> mkdir zk1 zk2 zk3 # 准备 zookeeper 服务 ID，每个服务器 ID 是不同的
shell> echo 1 > zk1/myid
shell> echo 2 > zk2/myid
shell> echo 3 > zk3/myic
shell> cp /usr/local/src/zookeeper-3.4.6/conf/zoo_sample.cfg /opt/zk1/zk1.cfg # 准备配置文件
# 第一个 zookeeper 配置文件
shell> grep "^[a-Z]" /opt/zk1/zk1.cfg
tickTime=2000 # 服务器和客户端的心跳维持间隔，间隔多久发送心跳 ，2000微秒等于2毫秒
initLimit=10 # 选举的时候的时间间隔是10次，10次 * 2000微秒 即20秒
syncLimit=5 # Leader 与 Follower 之间发送消息，请求和应答时间长度，最长不能超过多少个 tickTime 的时间长度
dataDir=/opt/zk1 # 数据保存目录
clientPort=2181 # 客户端连接的端口
server.1 = 192.168.10.101:2887:3887 # 集群端口和ID配置
server.2 = 192.168.10.101:2888:3888
server.3 = 192.168.10.101:2889:3889
配置第二个 zookeeper 服务
# 配置第二个 zookeeper 服务，每个服务对应不同的配置文件和数据目录
shell> cp /opt/zk1/zk1.cfg /opt/zk2/zk2.cfg
shell> vim /opt/zk2/zk2.cfg
shell> grep "^[a-Z]" /opt/zk2/zk2.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zk2
clientPort=2182
server.1 = 192.168.10.101:2887:3887
server.2 = 192.168.10.101:2888:3888
server.3 = 192.168.10.101:2889:3889
配置第三个 zookeeper 服务
# 配置第三个 zookeeper 服务，每个服务对应不同的配置文件和数据目录
shell> cp /opt/zk1/zk1.cfg /opt/zk3/zk3.cfg
shell> vim /opt/zk3/zk3.cfg
shell> grep "^[a-Z]" /opt/zk3/zk3.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zk3
clientPort=2183
server.1 = 192.168.10.101:2887:3887
server.2 = 192.168.10.101:2888:3888
server.3 = 192.168.10.101:2889:3889
参数详解
- tickTime
这个时间是作为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔，也就是每个 tickTime 时间就会发送一个心跳 - dataDir
顾名思义就是 Zookeeper 保存数据的目录，默认情况下，Zookeeper 将写数据的日志文件也保存在这个目录 - clientPort
这个端口就是客户端连接 Zookeeper 服务器的端口，Zookeeper 会监听这个端口，接受客户端的访问请求 - initLimit
这个配置项是用来配置 Zookeeper 接受客户端（这里所说的客户端不是用户连接 Zookeeper 服务器的客户端，而是 Zookeeper 服务器集群中连接至 Leader 的 Follower 服务器）初始化连接时最长能忍受多少个心跳时间间隔数，当已经超过 5 个心跳时间（也就是 tickTime）长度后 Zookeeper 服务器还没有收到客户端的返回信息，那么表明这个客户端连接失败，总的时间长度就是5*2000=10 秒 - syncLimit
这个配置项标识 Leader 与 Follower 之间发送消息，请求和应答时间长度，最长不能超过多少个 tickTime 的时间长度，总的时间长度就是2*2000=4 秒 - server.A=B : C : D
A是一个数字，表示这个是第几号服务器
B是此服务器 IP 地址
C是此服务器与集群 Leader 交换信息端口
D是选举用的通信端口，若是伪集群的配置方式，由于B都是一样，所以不同的 Zookeeper 示例通信端口号不一样，所以要给它们分配不同的端口号
启动 Zookeeper 服务
shell> /usr/local/zookeeper/bin/zkServer.sh start /opt/zk1/zk1.cfg
JMX enabled by default
Using config: /opt/zk1/zk1.cfg
Starting zookeeper ... STARTED
shell> /usr/local/zookeeper/bin/zkServer.sh start /opt/zk2/zk2.cfg
JMX enabled by default
Using config: /opt/zk2/zk2.cfg
Starting zookeeper ... STARTED
shell> /usr/local/zookeeper/bin/zkServer.sh start /opt/zk3/zk3.cfg
JMX enabled by default
Using config: /opt/zk3/zk3.cfg
Starting zookeeper ... STARTED
验证各 Zookeeper 启动完成
shell> ss -tnl | grep 2181
LISTEN 0 50 :::2181 :::*
shell> ss -tnl | grep 2181
LISTEN 0 50 :::2181 :::*
shell> ss -tnl | grep 2183
LISTEN 0 50 :::2183 :::*
查看各个 Zookeeper 节点的状态
shell> /usr/local/zookeeper/bin/zkServer.sh status /opt/zk1/zk1.cfg
JMX enabled by default
Using config: /opt/zk1/zk1.cfg
Mode: follower # 备
shell> /usr/local/zookeeper/bin/zkServer.sh status /opt/zk2/zk2.cfg
JMX enabled by default
Using config: /opt/zk2/zk2.cfg
Mode: leader # 主
shell> /usr/local/zookeeper/bin/zkServer.sh status /opt/zk3/zk3.cfg
JMX enabled by default
Using config: /opt/zk3/zk3.cfg
Mode: follower # 备
连接至 Zookeeper 节点
shell> /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.101:2181
Connecting to 192.168.10.101:2181
2020-07-16 13:59:24,793 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2020-07-16 13:59:24,797 [myid:] - INFO [main:Environment@100] - Client environment:host.name=192.168.10.101
2020-07-16 13:59:24,797 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_111
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-2.b15.el7_3.x86_64/jre
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf:
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2020-07-16 13:59:24,800 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-327.22.2.el7.x86_64
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2020-07-16 13:59:24,801 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/opt
2020-07-16 13:59:24,803 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=192.168.10.101:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@69d0a921
Welcome to ZooKeeper!
2020-07-16 13:59:24,846 [myid:] - INFO [main-SendThread(192.168.10.101:2181):ClientCnxn$SendThread@975] - Opening socket connection to server 192.168.10.101/192.168.10.101:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2020-07-16 13:59:24,934 [myid:] - INFO [main-SendThread(192.168.10.101:2181):ClientCnxn$SendThread@852] - Socket connection established to 192.168.10.101/192.168.10.101:2181, initiating session
2020-07-16 13:59:24,954 [myid:] - INFO [main-SendThread(192.168.10.101:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 192.168.10.101/192.168.10.101:2181, sessionid = 0x159a7a361160001, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: 192.168.10.101:2181(CONNECTED) 0]
Zookeeper 命令帮助
[zk: 192.168.10.101:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
stat path [watch]
set path data [version]
ls path [watch]
delquota [-n|-b] path
ls2 path [watch]
setAcl path acl
setquota -n|-b val path
history
redo cmdno
printwatches on|off
delete path [version]
sync path
listquota path
rmr path
get path [watch]
create [-s] [-e] path data acl
addauth scheme auth
quit
getAcl path
close
connect host:port
下载 Codis 2.0 版本
shell> go get -u -d github.com/CodisLabs/codis/tree/release2.0 # 下载 release 2.0 的包解压后编译然执行 mv codis-release2.0 codis 重命名
package github.com/CodisLabs/codis: no buildable Go source files in /opt/gopath /src/github.com/CodisLabs/codis
shell> cd /opt/gopath/src/github.com/CodisLabs/codis/
执行 make 进行编译
shell> make
fatal: Not a git repository (or any of the parent directories): .git
go build -i -o bin/codis-proxy ./cmd/proxy
go build -i -o bin/codis-config ./cmd/cconfig
make -j4 -C extern/redis-2.8.21/
make[1]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21'
cd src && make all
make[2]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/src'
rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html
(cd ../deps && make distclean)
make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps'
(cd hiredis && make clean) > /dev/null || true
(cd linenoise && make clean) > /dev/null || true
(cd lua && make clean) > /dev/null || true
(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true
(rm -f .make-*)
make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps'
(rm -f .make-*)
echo STD=-std=c99 -pedantic >> .make-settings
echo WARN=-Wall -W >> .make-settings
echo OPT=-O2 >> .make-settings
echo MALLOC=jemalloc >> .make-settings
echo CFLAGS= >> .make-settings
echo LDFLAGS= >> .make-settings
echo REDIS_CFLAGS= >> .make-settings
echo REDIS_LDFLAGS= >> .make-settings
echo PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -W -O2 -g -ggdb -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settings
echo PREV_FINAL_LDFLAGS= -g -ggdb -rdynamic >> .make-settings
(cd ../deps && make hiredis linenoise lua jemalloc)
make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps'
(cd hiredis && make clean) > /dev/null || true
(cd linenoise && make clean) > /dev/null || true
(cd lua && make clean) > /dev/null || true
(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true
(rm -f .make-*)
(echo "" > .make-cflags)
(echo "" > .make-ldflags)
MAKE hiredis
MAKE linenoise
cd hiredis && make static
MAKE lua
cd linenoise && make
cd lua/src && make all CFLAGS="-O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL " MYLDFLAGS="" AR="ar rcu"
make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src'
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lapi.o lapi.c
make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise'
cc -Wall -Os -g -c linenoise.c
MAKE jemalloc
cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS=""
make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis'
cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb net.c
make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise'
cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb hiredis.c
cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb sds.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lcode.o lcode.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldebug.o ldebug.c
cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb async.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldo.o ldo.c
ldo.c: In function ‘f_parser’:
ldo.c:496:7: warning: unused variable ‘c’ [-Wunused-variable]
int c = luaZ_lookahead(p->z);
^
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldump.o ldump.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lfunc.o lfunc.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lgc.o lgc.c
checking for xsltproc... /usr/bin/xsltproc
checking for gcc... gcc
ar rcs libhiredis.a net.o hiredis.o sds.o async.o
checking whether the C compiler works... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o llex.o llex.c
make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis'
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmem.o lmem.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lobject.o lobject.c
yes
checking for C compiler default output file name... a.out
checking for suffix of executables... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lopcodes.o lopcodes.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lparser.o lparser.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstate.o lstate.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstring.o lstring.c
checking whether we are cross compiling... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltable.o ltable.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltm.o ltm.c
no
checking for suffix of object files... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lundump.o lundump.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lvm.o lvm.c
o
checking whether we are using the GNU C compiler... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lzio.o lzio.c
yes
checking whether gcc accepts -g... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o strbuf.o strbuf.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o fpconv.o fpconv.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lauxlib.o lauxlib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lbaselib.o lbaselib.c
yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldblib.o ldblib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o liolib.o liolib.c
gcc -E
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmathlib.o lmathlib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loslib.o loslib.c
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltablib.o ltablib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstrlib.o lstrlib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loadlib.o loadlib.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o linit.o linit.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cjson.o lua_cjson.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_struct.o lua_struct.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cmsgpack.o lua_cmsgpack.c
yes
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_bit.o lua_bit.c
checking for sys/types.h... yes
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua.o lua.c
checking for sys/stat.h... yes
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o luac.o luac.c
cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o print.o print.c
checking for stdlib.h... ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o strbuf.o fpconv.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o lua_bit.o # DLL needs all object files
yes
ranlib liblua.a
checking for string.h... cc -o lua lua.o liblua.a -lm
cc -o luac luac.o print.o liblua.a -lm
liblua.a(loslib.o): In function `os_tmpname':
loslib.c:(.text+0x28c): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
yes
make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src'
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking size of void *... 8
checking size of int... 4
checking size of long... 8
checking size of intmax_t... 8
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking whether pause instruction is compilable... yes
checking whether SSE2 intrinsics is compilable... yes
checking for ar... ar
checking whether __attribute__ syntax is compilable... yes
checking whether compiler supports -fvisibility=hidden... yes
checking whether compiler supports -Werror... yes
checking whether tls_model attribute is compilable... no
checking for a BSD-compatible install... /usr/bin/install -c
checking for ranlib... ranlib
checking for ld... /usr/bin/ld
checking for autoconf... /usr/bin/autoconf
checking for memalign... yes
checking for valloc... yes
checking configured backtracing method... N/A
checking for sbrk... yes
checking whether utrace(2) is compilable... no
checking whether valgrind is compilable... no
checking STATIC_PAGE_SHIFT... 12
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for pthread_create in -lpthread... yes
checking for _malloc_thread_cleanup... no
checking for _pthread_mutex_init_calloc_cb... no
checking for TLS... yes
checking whether a program using ffsl is compilable... yes
checking whether atomic(9) is compilable... no
checking whether Darwin OSAtomic*() is compilable... no
checking whether to force 32-bit __sync_{add,sub}_and_fetch()... no
checking whether to force 64-bit __sync_{add,sub}_and_fetch()... no
checking whether Darwin OSSpin*() is compilable... no
checking for stdbool.h that conforms to C99... yes
checking for _Bool... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating doc/html.xsl
config.status: creating doc/manpages.xsl
config.status: creating doc/jemalloc.xml
config.status: creating include/jemalloc/jemalloc_macros.h
config.status: creating include/jemalloc/jemalloc_protos.h
config.status: creating include/jemalloc/internal/jemalloc_internal.h
config.status: creating test/test.sh
config.status: creating test/include/test/jemalloc_test.h
config.status: creating config.stamp
config.status: creating bin/jemalloc.sh
config.status: creating include/jemalloc/jemalloc_defs.h
config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h
config.status: creating test/include/test/jemalloc_test_defs.h
config.status: executing include/jemalloc/internal/private_namespace.h commands
config.status: executing include/jemalloc/internal/private_unnamespace.h commands
config.status: executing include/jemalloc/internal/public_symbols.txt commands
config.status: executing include/jemalloc/internal/public_namespace.h commands
config.status: executing include/jemalloc/internal/public_unnamespace.h commands
config.status: executing include/jemalloc/internal/size_classes.h commands
config.status: executing include/jemalloc/jemalloc_protos_jet.h commands
config.status: executing include/jemalloc/jemalloc_rename.h commands
config.status: executing include/jemalloc/jemalloc_mangle.h commands
config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands
config.status: executing include/jemalloc/jemalloc.h commands
===============================================================================
jemalloc version : 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340
library revision : 1
CC : gcc
CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT
CFLAGS : -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -fvisibility=hidden
LDFLAGS :
EXTRA_LDFLAGS :
LIBS : -lpthread
RPATH_EXTRA :
XSLTPROC : /usr/bin/xsltproc
XSLROOT :
PREFIX : /usr/local
BINDIR : /usr/local/bin
INCLUDEDIR : /usr/local/include
LIBDIR : /usr/local/lib
DATADIR : /usr/local/share
MANDIR : /usr/local/share/man
srcroot :
abs_srcroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/
objroot :
abs_objroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/
JEMALLOC_PREFIX : je_
JEMALLOC_PRIVATE_NAMESPACE
: je_
install_suffix :
autogen : 0
experimental : 1
cc-silence : 1
debug : 0
code-coverage : 0
stats : 1
prof : 0
prof-libunwind : 0
prof-libgcc : 0
prof-gcc : 0
tcache : 1
fill : 1
utrace : 0
valgrind : 0
xmalloc : 0
mremap : 0
munmap : 0
dss : 0
lazy_lock : 0
tls : 1
===============================================================================
cd jemalloc && make CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" lib/libjemalloc.a
make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc'
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/jemalloc.o src/jemalloc.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/arena.o src/arena.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/atomic.o src/atomic.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/base.o src/base.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/bitmap.o src/bitmap.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk.o src/chunk.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_dss.o src/chunk_dss.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_mmap.o src/chunk_mmap.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ckh.o src/ckh.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ctl.o src/ctl.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/extent.o src/extent.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/hash.o src/hash.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/huge.o src/huge.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mb.o src/mb.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mutex.o src/mutex.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/prof.o src/prof.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/quarantine.o src/quarantine.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/rtree.o src/rtree.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/stats.o src/stats.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tcache.o src/tcache.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/util.o src/util.c
gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tsd.o src/tsd.c
ar crus lib/libjemalloc.a src/jemalloc.o src/arena.o src/atomic.o src/base.o src/bitmap.o src/chunk.o src/chunk_dss.o src/chunk_mmap.o src/ckh.o src/ctl.o src/extent.o src/hash.o src/huge.o src/mb.o src/mutex.o src/prof.o src/quarantine.o src/rtree.o src/stats.o src/tcache.o src/util.o src/tsd.o
make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc'
make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps'
CC adlist.o
CC ae.o
CC anet.o
CC dict.o
CC redis.o
CC sds.o
CC zmalloc.o
CC lzf_c.o
CC lzf_d.o
CC pqsort.o
CC zipmap.o
CC sha1.o
CC ziplist.o
CC release.o
CC networking.o
CC util.o
CC object.o
CC db.o
CC replication.o
CC rdb.o
CC t_string.o
CC t_list.o
CC t_set.o
CC t_zset.o
CC t_hash.o
CC config.o
CC aof.o
CC pubsub.o
CC multi.o
CC debug.o
CC sort.o
CC intset.o
CC syncio.o
CC migrate.o
CC endianconv.o
CC slowlog.o
CC scripting.o
CC bio.o
CC rio.o
CC rand.o
CC memtest.o
CC crc64.o
CC crc32.o
CC bitops.o
CC sentinel.o
CC notify.o
CC setproctitle.o
CC hyperloglog.o
CC latency.o
CC sparkline.o
CC slots.o
CC redis-cli.o
CC redis-benchmark.o
CC redis-check-dump.o
CC redis-check-aof.o
LINK redis-server
LINK redis-benchmark
LINK redis-check-aof
LINK redis-check-dump
INSTALL redis-sentinel
LINK redis-cli
Hint: It's a good idea to run 'make test' 😉
make[2]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/src'
make[1]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21'
执行测试
shell> make gotest
go test ./pkg/... ./cmd/...
ok github.com/CodisLabs/codis/pkg/models 16.548s
ok github.com/CodisLabs/codis/pkg/proxy 9.212s
ok github.com/CodisLabs/codis/pkg/proxy/redis 1.942s
ok github.com/CodisLabs/codis/pkg/proxy/router 0.650s
? github.com/CodisLabs/codis/pkg/utils [no test files]
? github.com/CodisLabs/codis/pkg/utils/assert [no test files]
? github.com/CodisLabs/codis/pkg/utils/atomic2 [no test files]
ok github.com/CodisLabs/codis/pkg/utils/bytesize 0.002s
? github.com/CodisLabs/codis/pkg/utils/errors [no test files]
? github.com/CodisLabs/codis/pkg/utils/log [no test files]
? github.com/CodisLabs/codis/pkg/utils/trace [no test files]
? github.com/CodisLabs/codis/cmd/cconfig [no test files]
? github.com/CodisLabs/codis/cmd/proxy [no test files]
执行全部指令后，会在 bin 文件夹内生成 codis-config、codis-proxy、codis-server 三个可执行文件，另 bin/assets 文件夹是 codis-config 的 dashboard http 服务需要的前端资源，需要和 codis-config 放置在同一文件夹下
shell> ls bin/
assets
codis-config #配置命令，实现初始化迁移等功能
codis-proxy #代理
codis-server #codis 开发的codis server
可执行文件的使用方式
./bin/codis-config --help
usage: codis-config [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>]
<command> [<args>...]
options:
-c #配置文件地址
-L #日志输出文件地址
--log-level=<loglevel> #输出日志级别 (debug < info (default) < warn < error < fatal)
commands:
server #redis 服务器组管理
slot #slot 管理
dashboard #启动 dashboard 服务
action #事件管理 (目前只有删除历史事件的日志)
proxy #proxy 管理
默认启动的会读取当前目录的 config.ini 文件，编辑如下
dashboard 的配置文件
shell> vim config.ini
shell> grep "^[a-Z]" config.ini
coordinator=zookeeper
zk=192.168.10.101:2181,192.168.10.101:2182,192.168.10.101:2183 # zookeeper 的地址
product=mycodis # 项目名称
dashboard_addr=192.168.10.101:8090
password=123456 # 给 redis 设置的连接密码
backend_ping_period=5
session_max_timeout=1800
session_max_bufsize=131072
session_max_pipeline=1024
zk_session_timeout=30000
proxy_id=proxy_1 # 每个 ID 不一样
准备两个 proxy 的配置文件
shell> grep "^[a-Z]" config-1.ini # proxy1 配置文件
coordinator=zookeeper
zk=192.168.10.101:2181,192.168.10.101:2182,192.168.10.101:2183
product=mycodis
dashboard_addr=192.168.10.101:8090
password=123456
backend_ping_period=5
session_max_timeout=1800
session_max_bufsize=131072
session_max_pipeline=1024
zk_session_timeout=30000
proxy_id=proxy_1
shell> grep "^[a-Z]" config-2.ini # proxy2 配置文件
coordinator=zookeeper
zk=192.168.10.101:2181,192.168.10.101:2182,192.168.10.101:2183
product=mycodis
dashboard_addr=192.168.10.101:8090
password=123456
backend_ping_period=5
session_max_timeout=1800
session_max_bufsize=131072
session_max_pipeline=1024
zk_session_timeout=30000
proxy_id=proxy_2
启动 dashboard
shell> nohup ./bin/codis-config dashboard --addr 192.168.10.101:8090 --http-log /var/log/codis-dashboard.log > /var/log/codis.log &
查看启动日志
shell> tail /var/log/codis-dashboard.log
[martini]2017/01/16 15:37:05 listening on 192.168.10.101:8090 (development)
shell> tail /var/log/codis.log
2020-07-16 15:37:05 dashboard.go:160: [INFO] dashboard listening on addr: 192.168.10.101:8090
2020-07-16 15:37:05 dashboard.go:143: [INFO] dashboard node created: /zk/codis/db_mycodis/dashboard, {"addr": "192.168.10.101:8090", "pid": 33952}
2020-07-16 15:37:05 dashboard.go:144: [WARN] ********** Attention **********
2020-07-16 15:37:05 dashboard.go:145: [WARN] You should use `kill {pid}` rather than `kill -9 {pid}` to stop me,
2020-07-16 15:37:05 dashboard.go:146: [WARN] or the node resisted on zk will not be cleaned when I'm quiting and you must remove it manually
2020-07-16 15:37:05 dashboard.go:147: [WARN] *******************************
若报错，比如 zookeeper 已存在，需要进入 zookeeper，并使用 rmr /aa/bb/cc 删除之前的 dashboard 路径，然后再重新启动即可
[zk: 192.168.10.101:2181(CONNECTED) 2] ls /
[zk, zookeeper]
[zk: 192.168.10.101:2181(CONNECTED) 3] ls /zk
[codis]
[zk: 192.168.10.101:2181(CONNECTED) 4] ls /zkcodis
Node does not exist: /zkcodis
[zk: 192.168.10.101:2181(CONNECTED) 5] ls /zk/codis
[db_mycodis]
[zk: 192.168.10.101:2181(CONNECTED) 6] ls /zk/codis/db_mycodis
[migrate_tasks, dashboard]
[zk: 192.168.10.101:2181(CONNECTED) 7] ls /zk/codis/db_mycodis/dashboard
[]
[zk: 192.168.10.101:2181(CONNECTED) 8] rmr /zk/codis/db_mycodis/dashboard # 此步骤谨慎操作，后果自负！
[]
访问管理界面
示例
