分布式协议

go语言版本的Gossip协议包（memberlist）的使用

2020-01-01  本文已影响0人  zhangzhifei

说明

由于工作的契机，最近学习了下Gossip，以及go语言的实现版本HashiCorp/memberlist。网上有个最基本的memberlist使用的example，在下边的链接中，感兴趣可以按照文档运行下感受感受。本文主要讲解memberlist v0.1.5 的使用细节。

Gossip

Gossip是最终一致性协议，是目前性能最好，容错性最好的分布式协议。目前Prometheus的告警组件alertmanager、redis、s3、区块链等项目都有使用Gossip。本文不介绍Gossip原理，大家自行谷歌。

example

简单的几步即可搭建gossip集群

1、new个配置文件
c := memberlist.DefaultLocalConfig()
2、创建gossip网络
m, err := memberlist.Create(c)
3、将节点加入到集群
m.Join(parts)
4、实现delegate接口
5、将我们需要同步的数据加到广播队列
broadcasts.QueueBroadcast(userdate)

感谢已经有网友为我们实现了一个example（https://github.com/asim/memberlist
）。

配置文件

  1. 我们与memberlist交互就是这个Config 配置文件，里面包括基本配置BindAddr 、BindPort ，优化配置GossipInterval、 GossipNodes，代理接口Delegate、EventDelegate。memberlist给了我们三个默认配置，我们不做任何修改，直接创建默认配置即可以简单的跑启一个gossip集群。局域网络的配置DefaultLANConfig，外网DefaultWANConfig，本地DefaultLocalConfig，后两者都是在DefaultLANConfig
    基础上修改，区别是根据网速，调整了gossip interval、tcptimeout超时时间等。
  2. 接下来主要介绍下Config的各个配置项，以便大家在使用memberlist时，能够知道哪个参数应该修改，哪个代理需要实现。
type Config struct {
    // The name of this node. This must be unique in the cluster.
    Name string

    // Transport is a hook for providing custom code to communicate with
    // other nodes. If this is left nil, then memberlist will by default
    // make a NetTransport using BindAddr and BindPort from this structure.
    Transport Transport

    // Configuration related to what address to bind to and ports to
    // listen on. The port is used for both UDP and TCP gossip. It is
    // assumed other nodes are running on this port, but they do not need
    // to.
    BindAddr string
    BindPort int

    // Configuration related to what address to advertise to other
    // cluster members. Used for nat traversal.
    AdvertiseAddr string
    AdvertisePort int

    // ProtocolVersion is the configured protocol version that we
    // will _speak_. This must be between ProtocolVersionMin and
    // ProtocolVersionMax.
    ProtocolVersion uint8

    // TCPTimeout is the timeout for establishing a stream connection with
    // a remote node for a full state sync, and for stream read and write
    // operations. This is a legacy name for backwards compatibility, but
    // should really be called StreamTimeout now that we have generalized
    // the transport.
    TCPTimeout time.Duration

    /* IndirectChecks is the number of nodes that will be asked to perform
     an indirect probe of a node in the case a direct probe fails. Memberlist
     waits for an ack from any single indirect node, so increasing this
     number will increase the likelihood that an indirect probe will succeed
     at the expense of bandwidth. */
    IndirectChecks int

    // RetransmitMult is the multiplier for the number of retransmissions
    // that are attempted for messages broadcasted over gossip. The actual
    // count of retransmissions is calculated using the formula:
    //
    //   Retransmits = RetransmitMult * log(N+1)
    //
    // This allows the retransmits to scale properly with cluster size. The
    // higher the multiplier, the more likely a failed broadcast is to converge
    // at the expense of increased bandwidth.
    RetransmitMult int

    // SuspicionMult is the multiplier for determining the time an
    // inaccessible node is considered suspect before declaring it dead.
    // The actual timeout is calculated using the formula:
    //
    //   SuspicionTimeout = SuspicionMult * log(N+1) * ProbeInterval
    //
    // This allows the timeout to scale properly with expected propagation
    // delay with a larger cluster size. The higher the multiplier, the longer
    // an inaccessible node is considered part of the cluster before declaring
    // it dead, giving that suspect node more time to refute if it is indeed
    // still alive.
    SuspicionMult int

    // SuspicionMaxTimeoutMult is the multiplier applied to the
    // SuspicionTimeout used as an upper bound on detection time. This max
    // timeout is calculated using the formula:
    //
    // SuspicionMaxTimeout = SuspicionMaxTimeoutMult * SuspicionTimeout
    //
    // If everything is working properly, confirmations from other nodes will
    // accelerate suspicion timers in a manner which will cause the timeout
    // to reach the base SuspicionTimeout before that elapses, so this value
    // will typically only come into play if a node is experiencing issues
    // communicating with other nodes. It should be set to a something fairly
    // large so that a node having problems will have a lot of chances to
    // recover before falsely declaring other nodes as failed, but short
    // enough for a legitimately isolated node to still make progress marking
    // nodes failed in a reasonable amount of time.
    SuspicionMaxTimeoutMult int

    // PushPullInterval is the interval between complete state syncs.
    // Complete state syncs are done with a single node over TCP and are
    // quite expensive relative to standard gossiped messages. Setting this
    // to zero will disable state push/pull syncs completely.
    //
    // Setting this interval lower (more frequent) will increase convergence
    // speeds across larger clusters at the expense of increased bandwidth
    // usage.
    PushPullInterval time.Duration

    // ProbeInterval and ProbeTimeout are used to configure probing
    // behavior for memberlist.
    //
    // ProbeInterval is the interval between random node probes. Setting
    // this lower (more frequent) will cause the memberlist cluster to detect
    // failed nodes more quickly at the expense of increased bandwidth usage.
    //
    // ProbeTimeout is the timeout to wait for an ack from a probed node
    // before assuming it is unhealthy. This should be set to 99-percentile
    // of RTT (round-trip time) on your network.
    ProbeInterval time.Duration
    ProbeTimeout  time.Duration

    // DisableTcpPings will turn off the fallback TCP pings that are attempted
    // if the direct UDP ping fails. These get pipelined along with the
    // indirect UDP pings.
    DisableTcpPings bool

    // AwarenessMaxMultiplier will increase the probe interval if the node
    // becomes aware that it might be degraded and not meeting the soft real
    // time requirements to reliably probe other nodes.
    AwarenessMaxMultiplier int

    // GossipInterval and GossipNodes are used to configure the gossip
    // behavior of memberlist.
    //
    // GossipInterval is the interval between sending messages that need
    // to be gossiped that haven't been able to piggyback on probing messages.
    // If this is set to zero, non-piggyback gossip is disabled. By lowering
    // this value (more frequent) gossip messages are propagated across
    // the cluster more quickly at the expense of increased bandwidth.
    //
    // GossipNodes is the number of random nodes to send gossip messages to
    // per GossipInterval. Increasing this number causes the gossip messages
    // to propagate across the cluster more quickly at the expense of
    // increased bandwidth.
    //
    // GossipToTheDeadTime is the interval after which a node has died that
    // we will still try to gossip to it. This gives it a chance to refute.
    GossipInterval      time.Duration
    GossipNodes         int
    GossipToTheDeadTime time.Duration

    // GossipVerifyIncoming controls whether to enforce encryption for incoming
    // gossip. It is used for upshifting from unencrypted to encrypted gossip on
    // a running cluster.
    GossipVerifyIncoming bool

    // GossipVerifyOutgoing controls whether to enforce encryption for outgoing
    // gossip. It is used for upshifting from unencrypted to encrypted gossip on
    // a running cluster.
    GossipVerifyOutgoing bool

    // EnableCompression is used to control message compression. This can
    // be used to reduce bandwidth usage at the cost of slightly more CPU
    // utilization. This is only available starting at protocol version 1.
    EnableCompression bool

    // SecretKey is used to initialize the primary encryption key in a keyring.
    // The primary encryption key is the only key used to encrypt messages and
    // the first key used while attempting to decrypt messages. Providing a
    // value for this primary key will enable message-level encryption and
    // verification, and automatically install the key onto the keyring.
    // The value should be either 16, 24, or 32 bytes to select AES-128,
    // AES-192, or AES-256.
    SecretKey []byte

    // The keyring holds all of the encryption keys used internally. It is
    // automatically initialized using the SecretKey and SecretKeys values.
    Keyring *Keyring

    // Delegate and Events are delegates for receiving and providing
    // data to memberlist via callback mechanisms. For Delegate, see
    // the Delegate interface. For Events, see the EventDelegate interface.
    //
    // The DelegateProtocolMin/Max are used to guarantee protocol-compatibility
    // for any custom messages that the delegate might do (broadcasts,
    // local/remote state, etc.). If you don't set these, then the protocol
    // versions will just be zero, and version compliance won't be done.
    Delegate                Delegate
    DelegateProtocolVersion uint8
    DelegateProtocolMin     uint8
    DelegateProtocolMax     uint8
    Events                  EventDelegate
    Conflict                ConflictDelegate
    Merge                   MergeDelegate
    Ping                    PingDelegate
    Alive                   AliveDelegate

    // DNSConfigPath points to the system's DNS config file, usually located
    // at /etc/resolv.conf. It can be overridden via config for easier testing.
    DNSConfigPath string

    // LogOutput is the writer where logs should be sent. If this is not
    // set, logging will go to stderr by default. You cannot specify both LogOutput
    // and Logger at the same time.
    LogOutput io.Writer

    // Logger is a custom logger which you provide. If Logger is set, it will use
    // this for the internal logger. If Logger is not set, it will fall back to the
    // behavior for using LogOutput. You cannot specify both LogOutput and Logger
    // at the same time.
    Logger *log.Logger

    // Size of Memberlist's internal channel which handles UDP messages. The
    // size of this determines the size of the queue which Memberlist will keep
    // while UDP messages are handled.
    HandoffQueueDepth int

    // Maximum number of bytes that memberlist will put in a packet (this
    // will be for UDP packets by default with a NetTransport). A safe value
    // for this is typically 1400 bytes (which is the default). However,
    // depending on your network's MTU (Maximum Transmission Unit) you may
    // be able to increase this to get more content into each gossip packet.
    // This is a legacy name for backward compatibility but should really be
    // called PacketBufferSize now that we have generalized the transport.
    UDPBufferSize int

    // DeadNodeReclaimTime controls the time before a dead node's name can be
    // reclaimed by one with a different address or port. By default, this is 0,
    // meaning nodes cannot be reclaimed this way.
    DeadNodeReclaimTime time.Duration
}
配置项解释
代理

最后

哪里有问题，还请大家多多指正

参考

https://www.consul.io/docs/internals/gossip.html
https://en.wikipedia.org/wiki/Gossip_protocol
https://github.com/asim/memberlist
https://github.com/hashicorp/memberlist
https://zhuanlan.zhihu.com/p/41228196

上一篇下一篇

猜你喜欢

热点阅读