分布式全局唯一ID生成

2019-10-08  本文已影响0人  Audience0

分布式ID生成系统
要求:

  1. 全局唯一,不能重复,(基本要求);
  2. 递增,下一个ID大于上一个ID;(某些需求)
  3. 信息安全,非连续ID,避免恶意用户/竞争对手发现ID规则,从而猜出下一个ID或者根据ID总量猜出业务总量;
  4. 高可用,不能故障,可用性4个9或者5个9;(99.99%、99.999%)
  5. 高QPS,性能不能太差,否则容易造成线程堵塞;
  6. 平均延迟尽可能低

方式:

1.UUID:
UUID.randomUUID()
UUID太长,很多场景不适用;
有些场景希望id是数字的,UUID就不适用;
可读性不好;

2.数据库自增ID
auto_increment (mysql)
ID生成依赖数据库单机的读写性能;(高并发条件下性能不是很好)
对数据库依赖较大,数据库易发生性能瓶颈问题;

3.Redis方案
通过Redis原子操作命令INCR和INCRBY(redis自增)实现递增,同时可使用Redis集群提高吞吐量,集群后每台Redis的初始值为1,2,3,4,5,步长为5;
A:1,6,11,16,21
B:2,7,12,17,22
C:3,8,13,18,23
D:4,9,14,19,24
E:5,10,15,20,25
该方案是不错的;

4.Twiitter的snowflake算法
https://github.com/twitter/snowflake

5.MongoDB的ObjectID

6.zookeeper
方案一:通过持久顺序节点实现;
方案二:通过节点版本号;


/**
 * zookeeper
 * 方案一:通过有序节点获取全局唯一的ID
 */
public class IdGenerate {

    private static final String ID_NODE = "/NODE/zl";
    public static void main(String[] args) throws Exception {
        IdGenerate idGenerate = new IdGenerate();
        idGenerate.runThread();
    }

    /**
     * 通过zookeeper有序节点来生成唯一ID
     * @return
     * @throws Exception
     */
    private String idGen() throws Exception {
        CuratorClient curatorClient = new CuratorClient();

        if (null == curatorClient.getClient().checkExists().forPath(ID_NODE)){
            String s = curatorClient.getClient().create()
                    .creatingParentsIfNeeded()
                    .withMode(CreateMode.PERSISTENT_SEQUENTIAL)
                    .forPath(ID_NODE);

            //删除创建的节点,节省zookeeper空间
            curatorClient.getClient().delete().forPath(s);
            return s.substring(ID_NODE.length());
        }
        return "";
    }

    private CountDownLatch countDownLatch = new CountDownLatch(1);
    private void runThread() throws InterruptedException {

        long awaitTime = 5*1000;

        ExecutorService service = Executors.newFixedThreadPool(16);

        for(int i=0 ;i<16; i++){

            Thread.sleep(50);

            service.submit(()->{
                try {
                    //等到所有线程均提交任务后,执行业务代码
                    countDownLatch.await();
                    System.out.println(idGen());
                } catch (Exception e) {
                    e.printStackTrace();
                }
            });
        }

        countDownLatch.countDown();

        try{
            service.shutdown();
            if (!service.awaitTermination(awaitTime, TimeUnit.MILLISECONDS)){
                service.shutdownNow();
            };
        }catch (InterruptedException e){
            service.shutdownNow();
        }
    }
}


/**
 * zookeeper
 * 方案二:通过节点版本号;生成全局唯一ID
 */
public class IdGenerate02 {

    private static final String ID_NODE = "/NODE/zl";
    public static void main(String[] args) throws Exception {
        IdGenerate02 idGenerate = new IdGenerate02();
        idGenerate.runThread();
        //System.out.println(idGenerate.idGen());
    }

    /**
     * 通过节点版本号;生成全局唯一ID
     * @return
     * @throws Exception
     */
    private long idGen() throws Exception {
        CuratorClient curatorClient = new CuratorClient();
        if (curatorClient.getClient().checkExists().forPath(ID_NODE) == null){
            curatorClient.getClient().create()
                    .creatingParentsIfNeeded()
                    .withMode(CreateMode.PERSISTENT)
                    .forPath(ID_NODE);
        }
        Stat stat = curatorClient.getClient().setData().withVersion(-1).forPath(ID_NODE);
        return stat.getVersion();
    }

    private CountDownLatch countDownLatch = new CountDownLatch(1);
    private void runThread() throws InterruptedException {

        long awaitTime = 5*1000;

        ExecutorService service = Executors.newFixedThreadPool(16);

        for(int i=0 ;i<16; i++){

            Thread.sleep(50);

            service.submit(()->{
                try {
                    //等到所有线程均提交任务后,执行业务代码
                    countDownLatch.await();
                    System.out.println(idGen());
                } catch (Exception e) {
                    e.printStackTrace();
                }
            });
        }

        countDownLatch.countDown();

        try{
            service.shutdown();
            if (!service.awaitTermination(awaitTime, TimeUnit.MILLISECONDS)){
                service.shutdownNow();
            };
        }catch (InterruptedException e){
            service.shutdownNow();
        }
    }
}

上一篇 下一篇

猜你喜欢

热点阅读