《elastic-job》学习

2019-05-03  本文已影响0人  夏天嘚花花

概述

Elastic-Job是一个分布式调度解决方案，由两个相互独立的子项目Elastic-Job-Lite和Elastic-Job-Cloud组成。
官方文档地址:http://elasticjob.io/docs/elastic-job-lite/00-overview/
提供了如下功能

单机定时任务

通常我们在开发过程中或多或少会运用到定时任务。常用单机定时任务我们常用的包括:

单机定时任务的缺点

elastic-job的使用场景

elastic-job就是一个分布式的定时任务调度,我们能用它做什么?
例如有个需求需要在每天晚上统计当天的订单订单情况,单机模式下我们会写一段程序去处理，假如有10万个订单，处理成功需要耗时很长时间，且在机器瓶颈下可能会导致内存不足等情况。若使用elastic-job,我们则可以考虑将定时任务进行分片，让其分布在不同的机器上运行，提供可用性，减少单机失败带来的功能不可用。例如当下有2台机器，将任务分成4片，则每台机器处理其中2片任务。


image.png

使用实例初探

根据上图，实现将任务分成4片，分别在2台机器上执行
任务类

//具体执行的任务类
public class MyElasticJob implements SimpleJob {
    public final static Logger logger = LoggerFactory.getLogger(MyElasticJob.class);
    @Override
    public void execute(ShardingContext shardingContext) {
        int shardingTotalCount = shardingContext.getShardingTotalCount();
        String shardingParameter = shardingContext.getShardingParameter();
        //任务的分片项，从0开始递增例如有四个分片则序号为0~4
        int shardingItem = shardingContext.getShardingItem();
        //分片带的参数例如0=A,1=B,2=C,3=D
        String jobParamter = shardingContext.getJobParameter();
        logger.info("shardingItem =" + shardingItem + " , shardingTotalCount=" + shardingTotalCount + " , shardingParameter =" + shardingParameter + " ,jobParamter =" + jobParamter);
        //System.out.println("shardingItem =" + shardingItem + " , shardingTotalCount=" + shardingTotalCount + " , shardingParameter =" + shardingParameter + " ,jobParamter =" + jobParamter);
        switch (shardingItem){
            case  0:
                doLongJob("食品订单");
                break;
            case 1:
                doSmallJob("电脑订单");
                break;
            case 2:
                doLongJob("服装订单");
                break;
            case 3:
                doSmallJob("机械订单");
                break;
        }
    }

    public void doLongJob(String msg){
        logger.info("========长时间任务=======" + msg);
        try {
            TimeUnit.SECONDS.sleep(60);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    public void doSmallJob(String msg){
        logger.info("========短时间任务=======" + msg);
    }
}

配置类

public class FastDemo {
    private static CoordinatorRegistryCenter createRegistryCenter(){
      //elasticjob采用zookeeper进行任务的调度，根据抢主的方式实现同一时刻只有一个任务在一台机器上执行,保证不重复，这里配置zookeeper的地址
        CoordinatorRegistryCenter registryCenter = new ZookeeperRegistryCenter(new ZookeeperConfiguration("192.168.0.103:2181","elastic-job-demo"));
        registryCenter.init();
        return registryCenter;
    }

    private static LiteJobConfiguration createJobConfiguration() {
        //设置任务每15秒执行一次，一共分成4片，elastic-job会获取当前这个任务一共在多少台服务器上进行平均分配，例如我将war包分别放在了128，129机器上，任务总片为4,则按照elasticjob的默认分配策略，128将执行第0，1片任务，129执行第2，3片任务
        JobCoreConfiguration simpleCoreConfig = JobCoreConfiguration
                .newBuilder("demoSimpleJob", "0/20 * * * * ?", 4)
                .shardingItemParameters("0=A,1=B,2=C,3=D")
                .jobParameter("xuzy")
                .failover(true) //设置失效转移,当一台机器挂了以后他的分片会让其他台服务器执行
                .build();
        // 定义SIMPLE类型配置
        SimpleJobConfiguration simpleJobConfig = new SimpleJobConfiguration(simpleCoreConfig, MyElasticJob.class.getCanonicalName());
        // 定义Lite作业根配置
        LiteJobConfiguration simpleJobRootConfig = LiteJobConfiguration.newBuilder(simpleJobConfig).build();
        return simpleJobRootConfig;
    }

    public static void initJob(){
        new JobScheduler(createRegistryCenter(), createJobConfiguration()).init();
    }
}

ServletContextLTest类

public class ServletContextLTest implements ServletContextListener {
    @Override
    public void contextInitialized(ServletContextEvent sce) {
        FastDemo.initJob();
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
    }
}

web.xml

    <listener>
        <listener-class>com.xzy.elasticjob.servletListener.ServletContextLTest</listener-class>
    </listener>

将项目打包成war分别放在128,129上面分别观察日志
128日志

[root@server-1 bin]# taif -f /var/logs/elasticjob.log 
-bash: taif: command not found
[root@server-1 bin]# tail -f /var/logs/elasticjob.log 
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 1 threads.
  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

2019-04-27 02:12:03 [ INFO] - org.quartz.impl.StdSchedulerFactory -StdSchedulerFactory.java(1339) -Quartz scheduler 'demoSimpleJob' initialized from an externally provided properties instance.
2019-04-27 02:12:03 [ INFO] - org.quartz.impl.StdSchedulerFactory -StdSchedulerFactory.java(1343) -Quartz scheduler version: 2.2.1
2019-04-27 02:12:04 [ INFO] - org.quartz.core.QuartzScheduler -QuartzScheduler.java(575) -Scheduler demoSimpleJob_$_NON_CLUSTERED started.
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =2 , shardingTotalCount=4 , shardingParameter =C ,jobParamter =xuzy
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======服装订单
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =3 , shardingTotalCount=4 , shardingParameter =D ,jobParamter =xuzy
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======机械订单
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =2 , shardingTotalCount=4 , shardingParameter =C ,jobParamter =xuzy
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======服装订单
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =3 , shardingTotalCount=4 , shardingParameter =D ,jobParamter =xuzy
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======机械订单
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =2 , shardingTotalCount=4 , shardingParameter =C ,jobParamter =xuzy
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======服装订单
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =3 , shardingTotalCount=4 , shardingParameter =D ,jobParamter =xuzy
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======机械订单

129日志

[root@server-2 bin]# tail -f /var/logs/elasticjob.log 
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =1 , shardingTotalCount=4 , shardingParameter =B ,jobParamter =xuzy
2019-04-27 02:12:15 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======电脑订单
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =0 , shardingTotalCount=4 , shardingParameter =A ,jobParamter =xuzy
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======食品订单
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =1 , shardingTotalCount=4 , shardingParameter =B ,jobParamter =xuzy
2019-04-27 02:13:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======电脑订单
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =0 , shardingTotalCount=4 , shardingParameter =A ,jobParamter =xuzy
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======食品订单
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =1 , shardingTotalCount=4 , shardingParameter =B ,jobParamter =xuzy
2019-04-27 02:14:30 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(51) -========短时间任务=======电脑订单
2019-04-27 02:15:45 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =0 , shardingTotalCount=4 , shardingParameter =A ,jobParamter =xuzy
2019-04-27 02:15:45 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(42) -========长时间任务=======食品订单
2019-04-27 02:15:45 [ INFO] - com.xzy.elasticjob.MyElasticJob -MyElasticJob.java(23) -shardingItem =1 , shardingTotalCount=4 , shardingParameter =B ,jobParamter =xuzy

可以看到128,129分别处理了不同的数据分片，期间假如128服务器挂了，由于配置了失效转移，原来128的服装订单、机械订单任务将转移给129继续执行

elastic-job作业调度-zookeeper

elastic-job是通过zookeeper进行任务协调和故障转移。
当启动项目后会发现任务在zookeeper注册了节点，如下,首先在根据我们的配置创建了/elastic-job-demo/demoSimpleJob节点，对应的config,instances, leader, servers和sharding


image.png 
{
  "jobName": "demoSimpleJob", //任务名称
  "jobClass": "com.xzy.elasticjob.MyElasticJob", //具体执行类
  "jobType": "SIMPLE", //任务类型
  "cron": "0/20 * * * * ?", //任务实行时间corn
  "shardingTotalCount": 4, //总分片数
  "shardingItemParameters":  "0\u003dA,1\u003dB,2\u003dC,3\u003dD", //分片参数
  "jobParameter": "xuzy", //任务参数
  "failover": true, //是否失效转移
  "misfire": true,
  "description": "",
  "jobProperties": {
    "job_exception_handler": "com.dangdang.ddframe.job.executor.handler.impl.DefaultJobExceptionHandler",//默认的异常处理类
    "executor_service_handler": "com.dangdang.ddframe.job.executor.handler.impl.DefaultExecutorServiceHandler" //默认的作业处理线程池类
  },
  "monitorExecution": true,
  "maxTimeDiffSeconds": -1,
  "monitorPort": -1,
  "jobShardingStrategyClass": "",
  "reconcileIntervalMinutes": 10,
  "disabled": false, //作业是否禁止启动
  "overwrite": false //本地配置是否可覆盖注册中心配置
}

elastic-job-lite-console

elastic-job-lite-console是elasticjob提供的管理工具
安装方法

上一篇 下一篇

猜你喜欢

热点阅读