主从同步过程
2017-12-16 本文已影响0人
fred290
同步命令sync主要实现函数
syncCommand
{
日志：Slave %s asks for synchronization
/ * So the slave knows the new replid and offset to try a PSYNC later
* if the connection with the master is lost. */
函数：masterTryPartialResynchronization {
if (!server.repl_backlog || psync_offset < server.repl_backlog_off ||
psync_offset > (server.repl_backlog_off + server.repl_backlog_histlen))
{
日志："Unable to partial resync with slave %s for lack of backlog (Slave request was: %lld)."
if (psync_offset > server.master_repl_offset) {
日志："Warning: slave %s tried to PSYNC with an offset that is greater than the master replication offset."
}
走向全同步
}
/* If we reached this point, we are able to perform a partial resync:
* 1) Set client state to make it a slave.
* 2) Inform the client we can continue with +CONTINUE
* 3) Send the backlog data (from the offset to the end) to the slave. */
}
/ **********************************************************
开始全同步
**********************************************************/
/* Full resynchronization. */
/* Setup the slave as one waiting for BGSAVE to start. The following code
* paths will change the state if we handle the slave differently. */
如果是第一个连过来的slave，则创建repl_backlog
/* Create the replication backlog if needed. */
有BGSAVE进行的情形
/* CASE 1: BGSAVE is in progress, with disk target. */
/* CASE 2: BGSAVE is in progress, with socket target. */
/* CASE 3: There is no BGSAVE is progress. */
非 repl_diskless_sync情形下，没有AOF rewrite操作，进行Bgsave操作
startBgsaveForReplication(c->slave_capa) {
serverLog(LL_NOTICE,"Starting BGSAVE for SYNC with target: %s", socket_target ? "slaves sockets" :"disk");
rdbSaveBackground()
replicationSetupSlaveForFullResync(slave, getPsyncInitialOffset());
}
/ **********************************************************
在syncCommand函数中全同步结束，此时由主进程handle 子进程处理rdbsave的信号，
进一步创建事件，传输save的rdb数据
********************************************************** /
}
redis定时任务函数
int serverCron
{
/* Check if a background saving or AOF rewrite in progress terminated. */
if (server.rdb_child_pid != -1 || server.aof_child_pid != -1 || ldbPendingChildren())
{
backgroundSaveDoneHandler(exitcode,bysignal);
backgroundSaveDoneHandlerDisk(exitcode,bysignal);
日志：serverLog(LL_NOTICE, "Background saving terminated with success");
updateSlavesWaitingBgsave((!bysignal && exitcode == 0)
aeDeleteFileEvent(server.el,slave->fd,AE_WRITABLE);
if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE, sendBulkToSlave, slave) ==
AE_ERR)
{
freeClient(slave);
}
backgroundRewriteDoneHandler(exitcode,bysignal);
}
}
redis主从复制过程
先不解释replication buffer和replication backlog，而先看看redis主从复制的过程。
redis的主从复制分为两个阶段：
- 1）同步（sync rdb snapshot)：slave复制master的某时间点（t）的全量数据，t为master接收到slave的sync命令后执行rdb bgsave的时间点。2.8增加psync，支持full resync和partial resync命令。master发送rdb文件到slave，slave读取rdb把数据加载到内存。
- 2）命令传播（commands propagation）：同步时间点t后master上的数据更新到slave上， 发送的数据是redis的命令。
replication buffer的作用
redis的slave buffer（replication buffer，master端上）存放的数据是下面三个时间内所有的master数据更新操作。
- 1）master执行rdb bgsave产生snapshot的时间
- 2）master发送rdb到slave网络传输时间
- 3）slave load rdb文件把数据恢复到内存的时间
replication buffer太小会引发的问题：
replication buffer由client-output-buffer-limit slave设置，当这个值太小会导致主从复制链接断开。
- 1）当master-slave复制连接断开，server端会释放连接相关的数据结构。replication buffer中的数据也就丢失了，此时主从之间重新开始复制过程。
- 2）还有个更严重的问题，主从复制连接断开，导致主从上出现rdb bgsave和rdb重传操作无限循环。
