01.java增删改查性能速度优化

结论

• list查询改成map查询

Person有身份证字段：id
List<Person> list

找到list中id=xxx的person

解决：
先将list转为Map<String, Person> key为id,通过map.get(xx)提升性能

• list所有数据修改采用多线程

1.parallelStream.forEach()
缺点：main线程可能不会等待结束

2.Lists.partition(list, 1000);以1000大小分组
每个组使用线程池执行
优点：分组执行，每个线程执行各自的私有list,无锁的操作代价，最快

3.使用ConcurrentLinkedQueue 共享list，创建n个线程同时处理这个queue
缺点：多线程操作同一个queue，有锁的判断代价

3 demo

        log.info("111-1");
        
        //这种有4w条数据时反射啥的巨耗时
        
/*        List<CompanyRiskWarnVo> warnVos = dataRecords
                .stream()
                .map(dataRecord -> DataRecordTransform.toBean(dataRecord, CompanyRiskWarnVo.class))
                .collect(Collectors.toList());*/

        List<CompanyRiskWarnVo> warnVos = new ArrayList<>(dataRecords.size());

//按照上面3分组 ，每组一个线程执行，统一汇总结果
        List<List<DataRecord>> dataRecordParts = Lists.partition(dataRecords, 1000);

        List<Future<List<CompanyRiskWarnVo>>> recordFuture = new ArrayList<>();

        for (List<DataRecord> dataRecordPart : dataRecordParts) {
            Future<List<CompanyRiskWarnVo>> submit = threadPoolExecutor.submit(new Callable<List<CompanyRiskWarnVo>>() {
                @Override
                public List<CompanyRiskWarnVo> call() throws Exception {
                    try {
                        if (CollectionUtils.isNotEmpty(dataRecordPart)) {
                            return dataRecordPart.stream().map(dataRecord -> DataRecordTransform.toBean(dataRecord, CompanyRiskWarnVo.class)).collect(Collectors.toList());
                        }
                        return Collections.emptyList();
                    } catch (Exception e) {
                        return Collections.emptyList();
                    }
                }

            });

            recordFuture.add(submit);
        }

        for (Future<List<CompanyRiskWarnVo>> future : recordFuture) {
            List<CompanyRiskWarnVo> riskWarnVos = null;
            try {
                riskWarnVos = future.get();
            }  catch (Exception e) {
                log.warn("get transfer error, ", e);
                riskWarnVos = Collections.emptyList();
            }
            warnVos.addAll(riskWarnVos);
        }

stream.parallel

• 底层forkjoin多线程
• main线程会等待多线程结束
• ArrayList相对linkedList更适合stream遍历,容易随机拆分

[INFO  2022-12-07 10:26:53.907][main][:--]com.xiaomi.scf.risk.web.manager.CommonTest.lambda$parallel$2(CommonTest.java:258)[in parallel]
[INFO  2022-12-07 10:26:53.907][ForkJoinPool.commonPool-worker-1][:--]com.xiaomi.scf.risk.web.manager.CommonTest.lambda$parallel$2(CommonTest.java:258)[in parallel]
[INFO  2022-12-07 10:26:53.907][ForkJoinPool.commonPool-worker-2][:--]com.xiaomi.scf.risk.web.manager.CommonTest.lambda$parallel$2(CommonTest.java:258)[in parallel]
[INFO  2022-12-07 10:26:55.911][ForkJoinPool.commonPool-worker-3][:--]com.xiaomi.scf.risk.web.manager.CommonTest.lambda$parallel$2(CommonTest.java:258)[in parallel]
[INFO  2022-12-07 10:26:55.911][ForkJoinPool.commonPool-worker-5][:--]com.xiaomi.scf.risk.web.manager.CommonTest.lambda$parallel$2(CommonTest.java:258)[in parallel]
[INFO  2022-12-07 10:26:55.911][main][:--]com.xiaomi.scf.risk.web.manager.CommonTest.parallel(CommonTest.java:264)[main end]