docker部署elasticsearch + Kibana(6

2020-08-07 本文已影响0人架构师与哈苏

elasticsearch快速开始

docker run -d --name elasticsearch --net somenetwork -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.11

环境Linux

该vm.max_map_count内核设置需要至少设置为262144用于生产。取决于您的平台：

的Linux

该vm.max_map_count设置应该在/etc/sysctl.conf中永久设置：

vim /etc/sysctl.conf
# 这里设置多一点
vm.max_map_count=655360
# 刷新
sysctl -p

官网es6.8安装教程

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/docker.html

注意SpringBoot和ES的版本

img

Elasticsearch :docker-compose.yml

version: '3'
services:
  elasticsearch:
    image: elasticsearch:6.8.11
    container_name: myES
    environment:
      - TZ="Asia/Shanghai"
      - cluster.name=cluster1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /var/docker/elasticsearch/data:/usr/share/elasticsearch/data:rw
    ports:
      - 9200:9200
      - 9300:9300

Kibana:docker-compose.yml

官网参考

https://www.elastic.co/guide/en/kibana/6.8/docker.html#docker

version: '3'
services:
  kibana:
    image: kibana:6.8.11
    container_name: myKibana
    ports:
      - 5601:5601
    environment:
      TZ: "Asia/Shanghai"
      SERVER_NAME: kibana
      ELASTICSEARCH_HOSTS: http://elasticsearch.example.org
    volumes:
      - /var/docker/kibana/config:/usr/share/kibana/config

整合成一个docker-compose.yml

version: '3'
services:
  elasticsearch:
    image: elasticsearch:6.8.11
    container_name: myES
    restart: always
    environment:
      - TZ="Asia/Shanghai"
      - cluster.name=cluster1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /var/docker/elasticsearch/data:/usr/share/elasticsearch/data:rw
    ports:
      - 9200:9200
      - 9300:9300
  kibana:
    image: kibana:6.8.11
    container_name: myKibana
    restart: always
    ports:
      - 5601:5601
    environment:
      TZ: "Asia/Shanghai"
      SERVER_NAME: kibana
      ELASTICSEARCH_HOSTS: http://192.168.0.14:9200
    #volumes:
    #  - /var/docker/kibana/config:/usr/share/kibana/config

springboot配置

spring:
  data:
    elasticsearch:
      cluster-name: cluster1
      cluster-nodes: 192.168.0.14:9300
      repositories:
        enabled: true

pom.xml

<!--elasticsearch-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

PageUtil.java

/**
     * 获取Pageable
     * @param pageIndex 前端从1开始
     * @param pageSize
     * @return
     */
    static Pageable getPageable(Integer pageIndex, Integer pageSize, Sort sort){
        Pageable pageable = null;
        if (pageIndex != null && pageSize != null){
            if (pageIndex < 1) {
                pageIndex = 0;
            } else {
                pageIndex --;
            }

            if (pageSize < 1) {
                pageSize = 10;
            }

        } else {

            pageIndex = 0;
            pageSize = Integer.MAX_VALUE;

        }

        if (sort != null) {
            pageable = new PageRequest(pageIndex, pageSize, sort);
        } else {
            pageable = new PageRequest(pageIndex, pageSize);
        }

        return pageable;
    }

ProjectSearchRepository.java

package com.meeno.chemical.extend.search.repository;

import com.meeno.chemical.extend.search.entity.ProjectSearch;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

/**
 * @description: 项目搜索Repository
 * @author: Wzq
 * @create: 2020-08-06 15:23
 */
public interface ProjectSearchRepository extends ElasticsearchRepository<ProjectSearch,Long> {
}

SearchServiceImpl.java

package com.meeno.chemical.extend.search.service.impl;

import com.google.common.collect.Lists;
import com.meeno.chemical.extend.project.material.entity.Material;
import com.meeno.chemical.extend.search.entity.ProjectMaterialSearch;
import com.meeno.chemical.extend.search.entity.ProjectSearch;
import com.meeno.chemical.extend.search.repository.ProjectSearchRepository;
import com.meeno.chemical.extend.search.result.HighlightResultMapper;
import com.meeno.chemical.extend.search.service.ProjectSearchService;
import com.meeno.chemical.extend.search.view.ProjectSearchListView;
import com.meenoframework.util.PageData;
import com.meenoframework.util.PageUtil;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.SearchResultMapper;
import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
import org.springframework.data.elasticsearch.core.query.NativeSearchQuery;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.CollectionUtils;
import org.springframework.util.StringUtils;

import java.security.Key;
import java.util.ArrayList;
import java.util.List;

/**
 * @description: 项目搜索ServiceImpl
 * @author: Wzq
 * @create: 2020-08-06 15:25
 */
@Service
@Transactional(rollbackFor = Exception.class)
public class ProjectSearchServiceImpl implements ProjectSearchService {

    @Autowired
    private ProjectSearchRepository projectSearchRepository;

    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;


    @Override
    public void save(ProjectSearch projectSearch) {
        this.projectSearchRepository.save(projectSearch);
    }

    @Override
    public void update(Long projectId, String name) {
        ProjectSearch projectSearch = this.projectSearchRepository.findById(projectId).orElse(null);
        if(projectSearch != null){
            projectSearch.setProjectName(name);
            this.projectSearchRepository.save(projectSearch);
        }
    }

    @Override
    public void update(Long projectId, List<Material> materialSearchList) {
        ProjectSearch projectSearch = this.projectSearchRepository.findById(projectId).orElse(null);
        if(projectSearch != null){
            String materialsStr = "";
            if(!CollectionUtils.isEmpty(materialSearchList)){
                for (int i = 0; i < materialSearchList.size(); i++) {
                    if(i == materialSearchList.size()-1){
                        materialsStr += materialSearchList.get(i).getName();
                    }else{
                        materialsStr += materialSearchList.get(i).getName() + ",";
                    }
                }
            }
            projectSearch.setMaterialsStr(materialsStr);
            this.projectSearchRepository.save(projectSearch);
        }
    }

    @Override
    public void delete(Long id) {
        this.projectSearchRepository.deleteById(id);
    }

    @Override
    public ProjectSearch get(Long id) {
        ProjectSearch projectSearch = this.projectSearchRepository.findById(id).orElse(null);
        return projectSearch;
    }

    @Override
    public List<ProjectSearch> findAll() {
        Iterable<ProjectSearch> searchIterable = this.projectSearchRepository.findAll();
        List<ProjectSearch> list = Lists.newArrayList();
        searchIterable.forEach(search ->{
           list.add(search);
        });
        return list;
    }

    @Override
    public PageData<ProjectSearch> findPage(Integer pageIndex, Integer pageSize) {
        Sort.Order createTimeDesc = Sort.Order.desc("createTime");
        Sort sort = Sort.by(createTimeDesc);
        Pageable pageable = PageUtil.getPageable(pageIndex, pageSize, sort);

        Page<ProjectSearch> page = this.projectSearchRepository.findAll(pageable);

        PageData<ProjectSearch> pageData = new PageData<>(page.getTotalElements(),page.getContent());

        return pageData;
    }

    @Override
    public PageData<ProjectSearch> findPageKeyword(Integer pageIndex, Integer pageSize, String keywords) {
        Pageable pageable = PageUtil.getPageable(pageIndex, pageSize);

        //高亮
        String preTag = "<span style=‘color:red’>";
        String postTag = "</font>";

        // 构造查询条件
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
        //keyworks为空查询全部
        if(StringUtils.isEmpty(keywords)){
            Sort.Order createTimeDesc = Sort.Order.desc("createTime");
            Sort sort = Sort.by(createTimeDesc);
            pageable = PageUtil.getPageable(pageIndex, pageSize,sort);
            queryBuilder.withPageable(pageable);
        }else{
            queryBuilder.withQuery(QueryBuilders.multiMatchQuery(keywords,"projectName","materialsStr"))
                    .withHighlightFields(
                            new HighlightBuilder.Field("projectName").preTags(preTag).postTags(postTag),
                            new HighlightBuilder.Field("materialsStr").preTags(preTag).postTags(postTag)
                    )
                    .withPageable(pageable);

//对象嵌套查询
//queryBuilder.withQuery(QueryBuilders.matchQuery("materialSearchList.materialName",keywords))
//                    .withPageable(pageable);
        }

        NativeSearchQuery searchQuery = queryBuilder.build();

        Page<ProjectSearch> searchPage = this.elasticsearchTemplate.queryForPage(searchQuery, ProjectSearch.class, new HighlightResultMapper());

        //Page<ProjectSearch> searchPage = this.projectSearchRepository.search(searchQuery);

        PageData<ProjectSearch> pageData = new PageData<>(searchPage.getTotalElements(),searchPage.getContent());


        return pageData;
    }
}

高亮处理类 HighlightResultMapper.java

package com.meeno.chemical.extend.search.result;

import com.alibaba.fastjson.JSON;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.core.SearchResultMapper;
import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
import org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl;

import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

/**
 * @description:
 * @author: Wzq
 * @create: 2020-08-07 12:40
 */
public class HighlightResultMapper implements SearchResultMapper {

    @Override
    public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> clazz, Pageable pageable) {
        long totalHits = searchResponse.getHits().getTotalHits();
        List<T> list = new ArrayList<>();
        SearchHits hits = searchResponse.getHits();
        if (hits.getHits().length> 0) {
            for (SearchHit searchHit : hits) {
                Map<String, HighlightField> highlightFields = searchHit.getHighlightFields();
                T item = JSON.parseObject(searchHit.getSourceAsString(), clazz);
                Field[] fields = clazz.getDeclaredFields();
                for (Field field : fields) {
                    field.setAccessible(true);
                    if (highlightFields.containsKey(field.getName())) {
                        try {
                            field.set(item, highlightFields.get(field.getName()).fragments()[0].toString());
                        } catch (IllegalAccessException e) {
                            e.printStackTrace();
                        }
                    }
                }
                list.add(item);
            }
        }
        return new AggregatedPageImpl<>(list, pageable, totalHits);
    }

    /*@Override
    public <T> T mapSearchHit(SearchHit searchHit, Class<T> aClass) {
        return null;
    }*/

}

参考文档

https://blog.csdn.net/tianyaleixiaowu/article/details/77965257
https://blog.csdn.net/weter_drop/article/details/104581758

可视化工具 dejavu

elasticsearch可视化 dejavu
https://github.com/appbaseio/dejavu

docker run -p 1358:1358 -d appbaseio/dejavu

http.port: 9200
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

docker部署elasticsearch + Kibana(6

elasticsearch快速开始

环境Linux

官网es6.8安装教程

注意SpringBoot和ES的版本

Elasticsearch :docker-compose.yml

Kibana:docker-compose.yml

官网参考

整合成一个docker-compose.yml

springboot配置

pom.xml

PageUtil.java

ProjectSearchRepository.java

SearchServiceImpl.java

高亮处理类 HighlightResultMapper.java

参考文档

可视化工具 dejavu

猜你喜欢

热点阅读