Java对接geoserver实现图层组与图层发布
2023-06-28 本文已影响0人
蒋仁勇
最近一个项目遇到了需要对接geoserver的需求，要定时同步固定目录下的tif到geoserver中，并发布成图层组给前端使用
首先需要将固定目录mount 到服务器目录上，这里参考了这些博客
Linux创建并挂载NAS：win10 https://blog.csdn.net/cyf12345678/article/details/126592249
win7 https://www.haozhuangji.com/xtjc/155916801.htmlhttps://www.xitongzhijia.net/xtjc/20111207/2349.html
https://blog.csdn.net/cyf12345678/article/details/126592249
核心的命令其实就下面这些
//liunx安装挂载工具
yum install -y cifs-utils
//挂载共享目录
mount -t cifs -o username=username,password=password //192.168.1.139/layer/win_share/ /data/share_data/win_share
打开文件 CIFS和SMB类似
smb://user_name:password@server_name/.... \server_name...
webdav http://ip:5005/.....
ftp://ip/....
geoserver
api文档（英文）：https://docs.geoserver.org/stable/en/user/rest/index.html#rest
docker部署：https://www.bilibili.com/read/cv17617766/
docker pull kartoza/geoserver
启动脚本：docker run --restart=always -d -v /data/share_data:/etc/letsencrypt -p 31880:8080 --name geoserver kartoza/geoserver
注意mount之后要重新启动一下geoserver才会出现新挂载的目录：docker restart geoserver
admin的初始密码在日志里
image.png
添加用户：admin的初始密码在日志里，需要重新添加用户（别忘了给新增的用户加上角色）并设置密码
image.png
image.png
新建工作空间
image.png
新建存储仓库：tif文件需要先上传到部署geoserver的服务器的/data/share_data目录下
image.png
点击发布
图层可以使用
以上为基础设施的建设，下面是代码方面
架构.png
架构很简单，就是通过mount挂载电脑或者nas上的文件夹到服务器的data_share目录下，Java程序定时扫描该目录下的tif文件，再发布到geoserver服务上并拼装图层组的url给前端使用。
Maven引入geoserver-manager的sdk:
<dependency>
<groupId> nl.pdok</groupId>
<artifactId>geoserver-manager</artifactId>
<version>1.7.0-pdok2</version>
</dependency>
geoserver配置:
import it.geosolutions.geoserver.rest.GeoServerRESTManager;
import java.net.URL;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.annotation.Resource;
@Configuration
public class GeoServerConfig {
@Resource
private GeoServerProperties geoServerProperties;
@Bean(name = "geoServerRESTManager")
public GeoServerRESTManager geoServerRESTManager() {
try {
return new GeoServerRESTManager(new URL(geoServerProperties.getEndpoint()),
geoServerProperties.getUsername(), geoServerProperties.getPassword());
} catch (Exception ex) {
ex.printStackTrace();
return null;
}
}
}
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
@Data
@Configuration
@ConfigurationProperties(prefix = "spring.geoserver")
public class GeoServerProperties {
private String endpoint;
private String username;
private String password;
private String workSpaceName;
}
spring:
geoserver:
endpoint: http://${global.geoserver.ip}:${global.geoserver.port}/geoserver
username: username
password: password
workSpaceName: test
具体代码：其中需要注意的是我写了一个规则来根据文件的目录对应挂载主机的ip(需要这个ip+目录可以直接访问到对应主机上的文件)
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.jingansi.suitation.config.GeoServerProperties;
import com.jingansi.suitation.dao.mapper.LayerGroupMapper;
import com.jingansi.suitation.dao.mapper.LayerMapper;
import com.jingansi.suitation.dao.model.Layer;
import com.jingansi.suitation.dao.model.LayerGroup;
import com.jingansi.suitation.dao.model.Point;
import com.jingansi.suitation.enums.LayerGroupAffiliation;
import com.jingansi.suitation.exception.BizException;
import com.jingansi.suitation.model.LayerGroupDTO;
import com.jingansi.suitation.model.RangeReq;
import com.jingansi.suitation.service.GeoServerService;
import com.jingansi.suitation.util.FileUtil;
import it.geosolutions.geoserver.rest.GeoServerRESTManager;
import it.geosolutions.geoserver.rest.decoder.RESTCoverageStore;
import it.geosolutions.geoserver.rest.decoder.RESTLayerGroup;
import it.geosolutions.geoserver.rest.encoder.GSLayerEncoder;
import it.geosolutions.geoserver.rest.encoder.GSLayerGroupEncoder;
import it.geosolutions.geoserver.rest.encoder.coverage.GSImageMosaicEncoder;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.math.NumberUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.CollectionUtils;
import javax.annotation.Resource;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.util.*;
@Slf4j
@Service
public class GeoServerServiceImpl extends ServiceImpl<LayerMapper, Layer> implements GeoServerService {
@Resource
private GeoServerProperties geoServerProperties;
@Resource
private GeoServerRESTManager geoServerRESTManager;
@Resource
private LayerGroupMapper layerGroupMapper;
@Resource
private LayerMapper layerMapper;
@Value("${mount.base.shareContainerPath:/etc/letsencrypt/}")
private String shareContainerPath;
@Value("${mount.base.localPathRule:win_share:192.168.1.139,win_share2:192.168.1.14,nas_share:192.168.1.19}")
private String localPathRule;
@Override
public List<LayerGroupDTO> layerGroupList() {
List<LayerGroupDTO> layerGroups = new ArrayList<>();
LambdaQueryWrapper<LayerGroup> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(LayerGroup::getLayerGroupAffiliation, LayerGroupAffiliation.SAR);
List<LayerGroup> sarLayerGroups = layerGroupMapper.selectList(queryWrapper);
layerGroups.add(LayerGroupDTO.builder().layerGroupAffiliation(LayerGroupAffiliation.SAR.name()).count(sarLayerGroups.size()).data(sarLayerGroups).build());
return layerGroups;
}
@Override
public List<Layer> layerList(Long layerGroupId) {
LambdaQueryWrapper<Layer> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(Layer::getLayerGroupId, layerGroupId);
return layerMapper.selectList(queryWrapper);
}
/**
* 找出经纬度在范围里的图层
* 需要记录图层的上下左右四个点的经纬度
**/
@Override
public List<Layer> rangeList(RangeReq rangeReq) {
List<Layer> rangeLayers = new ArrayList<>();
LambdaQueryWrapper<Layer> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.in(Layer::getLayerGroupId, rangeReq.getLayerGroupIds());
//查询图层组的图层
List<Layer> groupLayers = layerMapper.selectList(queryWrapper);
groupLayers.forEach(layer -> {
List<Point> ps = new ArrayList<>();
ps.add(Point.builder().longitude(layer.getTopLeftLongitude()).latitude(layer.getTopLeftLatitude()).build());
ps.add(Point.builder().longitude(layer.getTopRightLongitude()).latitude(layer.getTopRightLatitude()).build());
ps.add(Point.builder().longitude(layer.getDownRightLongitude()).latitude(layer.getDownRightLatitude()).build());
ps.add(Point.builder().longitude(layer.getDownLeftLongitude()).latitude(layer.getDownLeftLatitude()).build());
//判断经纬度是否在图层内
if (containsLngLat(ps.toArray(new Point[4]), Point.builder().longitude(rangeReq.getLongitude()).latitude(rangeReq.getLatitude()).build())) {
rangeLayers.add(layer);
}
});
return rangeLayers;
}
/**
* 判断点是否在多边形内（该方法来自前端，不保证其正确性--部分图像中心点会出现不在范围内的情况(高纬度等偏斜情形)--要顺时针或者逆时针顺序）
*
* @param {*} points [{lng,lat}]
* @param {*} point {lng,lat}
* @returns
*/
public static Boolean containsLngLat(Point[] points, Point point) {
try {
// 数组长度
int length = points.length;
boolean c = false;
for (int i = 0, j = length - 1; i < length; ) {
if (
points[i].getLatitude() > point.getLatitude() != points[j].getLatitude() > point.getLatitude() &&
point.getLongitude() <
((points[j].getLongitude() - points[i].getLongitude()) * (point.getLatitude() - points[i].getLatitude())) /
(points[j].getLatitude() - points[i].getLatitude()) +
points[i].getLongitude()
) {
c = !c;
}
j = i;
i += 1;
}
return c;
} catch (Exception e) {
e.printStackTrace();
log.error("containsLngLat point:{} points:{}", point, points);
return false;
}
}
@Override
@Transactional(rollbackFor = Exception.class)
public Long addLayerGroup(LayerGroup layerGroup) {
try {
Date now = new Date();
List<Layer> layers = new ArrayList<>();
//处理图层
ArrayList<String> layersList = handleLayer(layerGroup, layers);
if (!CollectionUtils.isEmpty(layers)) {
//创建并发布图层组
createLayerGroup(layerGroup.getLayerGroupName(), layersList);
//默认图层组为SAR
if (StringUtils.isBlank(layerGroup.getLayerGroupAffiliation())) {
layerGroup.setLayerGroupAffiliation(LayerGroupAffiliation.SAR.name());
}
setLayerGroupUrlAndBbox(layerGroup);
layerGroup.setGmtCreate(now);
layerGroup.setGmtModified(now);
//保存图层组
layerGroupMapper.insert(layerGroup);
layers.forEach(layer -> {
layer.setLayerGroupId(layerGroup.getId());
layer.setGmtCreate(now);
layer.setGmtModified(now);
layer.setGmtCreateBy(layerGroup.getGmtCreateBy());
layer.setGmtModifiedBy(layerGroup.getGmtModifiedBy());
});
//保存图层
saveBatch(layers);
}
} catch (BizException e) {
log.error("addLayerGroup BizException error:" + JSON.toJSONString(layerGroup), e);
throw e;
} catch (Exception e) {
log.error("addLayerGroup error:" + JSON.toJSONString(layerGroup), e);
throw new BizException("新增图层失败");
}
return layerGroup.getId();
}
@Override
public ArrayList<String> handleLayer(LayerGroup layerGroup, List<Layer> layers) throws Exception {
//fileDirectory 处理
String fileDirectory = shareContainerPath + layerGroup.getLayerGroupDirectory();
//图层组名称如果为空则设置为入参处理后的目录的MD5值---前端必填字段，走不到这里
if (StringUtils.isBlank(layerGroup.getLayerGroupName())) {
layerGroup.setLayerGroupName(DigestUtils.md5Hex(fileDirectory));
}
File layerDirectory = FileUtil.getFile(fileDirectory);
String[] layerPaths = layerDirectory.list();
ArrayList<String> layersList = handleLayer(layers, fileDirectory, layerPaths);
if (CollectionUtils.isEmpty(layersList)) {
throw new BizException("新增图层失败：图层文件夹下未读取到tif图层");
}
return layersList;
}
@Override
public ArrayList<String> handleLayer(List<Layer> layers, String fileDirectory, String[] layerPaths) {
ArrayList<String> layersList = new ArrayList<>();
Map<String, String> pathIpMap = getPathIpMap();
if (layerPaths != null) {
for (String layerPath : layerPaths) {
if (StringUtils.isBlank(layerPath)) {
continue;
}
try {
if (layerPath.endsWith(".tif")) {
String layerName = layerPath.split(".tif")[0];
Layer layer = new Layer();
//解析配置文件中的目录与ip关系
String ip = "";
for (Map.Entry<String, String> entry : pathIpMap.entrySet()) {
if (fileDirectory.contains(entry.getKey())) {
ip = entry.getValue();
}
}
layer.setLayerTifUrl(ip + fileDirectory.replace(shareContainerPath, "") + layerPath);
layer.setLayerName(layerName);
layersList.add(geoServerProperties.getWorkSpaceName() + ":" + layerName);
//推送图层数据
publishTiffData(layerName, fileDirectory + "/" + layerPath);
layers.add(layer);
}
} catch (Exception e) {
log.error("图层解析失败", e);
}
}
}
return layersList;
}
/**
* 获取ip路径映射Map
*
* @return {@link Map}<{@link String}, {@link String}>
*/
@Override
public Map<String, String> getPathIpMap() {
String[] rules = localPathRule.split(";");
Map<String, String> pathIpMap = new HashMap<>(16);
for (String rule : rules) {
String[] pathIp = rule.split(",");
pathIpMap.put(pathIp[0], pathIp[1]);
}
return pathIpMap;
}
private void setLayerGroupUrlAndBbox(LayerGroup layerGroup) {
String layerGroupName = layerGroup.getLayerGroupName();
RESTLayerGroup restLayerGroup = geoServerRESTManager.getReader().getLayerGroup(geoServerProperties.getWorkSpaceName(), layerGroupName);
String nativeCrs = restLayerGroup.getCRS();
layerGroup.setLayer(geoServerProperties.getWorkSpaceName() + ":" + layerGroupName);
layerGroup.setBboxMinX(restLayerGroup.getMinX());
layerGroup.setBboxMinY(restLayerGroup.getMinY());
layerGroup.setBboxMaxX(restLayerGroup.getMaxX());
layerGroup.setBboxMaxY(restLayerGroup.getMaxY());
//返回图层预览wms地址
/*return geoServerProperties.getEndpoint() + "/" + geoServerProperties.getWorkSpaceName() + "/wms?service=WMS&version=1.1.0&request=GetMap&layers=" + geoServerProperties.getWorkSpaceName() + ":" + layerGroupName + "&bbox="
+ nativeMinX + "," + nativeMinY + "," + nativeMaxX + "," + nativeMaxY +
//todo 宽高写死了
"&width=768&height=330&srs=" + nativeCrs + "&styles=&format=application/openlayers";*/
//返回图层预览缓存gwc地址-wmts
layerGroup.setLayerGroupUrl(geoServerProperties.getEndpoint() + "/gwc/demo/" + geoServerProperties.getWorkSpaceName() + ":" + layerGroupName + "?gridSet="
+ nativeCrs + "&format=image/png");
}
/**
* 创建图层组
* @param layerGroupName 图层组名称
* @param layersList 图层名称队列：格式为WorkSpace：图层名称 eg： ja-test:tiff-test
* @return 是否成功
*/
private boolean createLayerGroup(String layerGroupName, ArrayList<String> layersList) {
GSLayerGroupEncoder gsLayerGroupEncoder = new GSLayerGroupEncoder();
gsLayerGroupEncoder.setWorkspace(geoServerProperties.getWorkSpaceName());
gsLayerGroupEncoder.setName(layerGroupName);
for (String layer : layersList) {
gsLayerGroupEncoder.addLayer(layer);
}
return geoServerRESTManager.getPublisher().createLayerGroup(geoServerProperties.getWorkSpaceName(), layerGroupName, gsLayerGroupEncoder);
}
/**
* 调用geoserver为图层组配置图层
*/
@Override
public boolean configureLayerGroup(String layerGroupName, ArrayList<String> layersList) {
GSLayerGroupEncoder gsLayerGroupEncoder = new GSLayerGroupEncoder();
gsLayerGroupEncoder.setWorkspace(geoServerProperties.getWorkSpaceName());
gsLayerGroupEncoder.setName(layerGroupName);
for (String layer : layersList) {
gsLayerGroupEncoder.addLayer(layer);
}
return geoServerRESTManager.getPublisher().configureLayerGroup(geoServerProperties.getWorkSpaceName(), layerGroupName, gsLayerGroupEncoder);
}
/**
* 发布tif格式图层
*
* @param storeName 目录名称/图层名称
* @param fileDirectory tif文件全路径
*/
private void publishTiffData(String storeName, String fileDirectory) throws Exception {
GSImageMosaicEncoder gsCoverageEncoder = new GSImageMosaicEncoder();
//设置坐标系
gsCoverageEncoder.setSRS("EPSG:4326");
gsCoverageEncoder.setName(storeName);
//todo 去斜拍照片的黑边，但是会导致图层发白，清晰度丢失，暂无解决办法（样式？）
gsCoverageEncoder.setInputTransparentColor("#000000");
GSLayerEncoder layerEncoder = new GSLayerEncoder();
RESTCoverageStore publish = geoServerRESTManager.getPublisher().publishExternalGeoTIFF(geoServerProperties.getWorkSpaceName(), storeName, new File(fileDirectory), gsCoverageEncoder, layerEncoder);
log.info("publishTiffData store:{} fileDirectory:{} publish (TIFF文件发布状态) : {}", storeName, fileDirectory, publish);
}
}
上面的LayerGroup和Layer实体类：
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
@TableName(value ="layer_group")
@Data
public class LayerGroup implements Serializable {
/**
*
*/
@TableId(type = IdType.AUTO)
private Long id;
/**
* 设备id
*/
private String deviceId;
/**
* 图层组名称
*/
private String layerGroupName;
/**
* 图层组所属：SAR/CCD
*/
private String layerGroupAffiliation;
/**
* 图层组bbox
*/
private Double bboxMinX;
/**
* 图层组bbox
*/
private Double bboxMinY;
/**
* 图层组bbox
*/
private Double bboxMaxX;
/**
* 图层组bbox
*/
private Double bboxMaxY;
/**
* 图层组路径
*/
private String layerGroupUrl;
/**
* 图层组存储目录
*/
private String layerGroupDirectory;
/**
* 创建时间
*/
private Date gmtCreate;
/**
* 更新时间
*/
private Date gmtModified;
/**
* 创建者id
*/
private String gmtCreateBy;
/**
* 更新者id
*/
private String gmtModifiedBy;
@TableField(exist = false)
private static final long serialVersionUID = 1L;
/**
* 图层组名称 workspace:name
*/
private String layer;
}
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
@TableName(value ="layer")
@Data
public class Layer implements Serializable {
/**
*
*/
@TableId(type = IdType.AUTO)
private Long id;
/**
* 所属图层组ID
*/
private Long layerGroupId;
/**
* 设备id
*/
private String deviceId;
/**
* 图层名称
*/
private String layerName;
/**
* 图层tif图片文件路径
*/
private String layerTifUrl;
/**
* 拓展字段
*/
private String layerExtend;
/**
* 创建时间
*/
private Date gmtCreate;
/**
* 更新时间
*/
private Date gmtModified;
/**
* 创建者id
*/
private String gmtCreateBy;
/**
* 更新者id
*/
private String gmtModifiedBy;
@TableField(exist = false)
private static final long serialVersionUID = 1L;
}
遇到的坑：
1.拼装的geoserver预览url加载很慢
image.png
这个问题困扰了我整整2天时间，中间各种百度和求助前同事，均无头绪。
尝试过加配置，集群，换交换机，甚至修改源码了，最后才发现问题出在了mount挂载这一步
原来受限于nas和服务器间的网线是百兆且是二手机械盘，导致nas往服务器copy文件的速度在1Mb/s-10Mb/s,而一张100多Mb的tif图片光传输都要10多秒，所以通过top看geoserver根本的资源消耗根本不大，后来我尝试直接把文件拷贝到服务器里请求速度直接起飞到几毫秒，然后就是更换nas网线为千兆线，硬盘换位ssd后速度达到几秒一次
2.原图发布斜拍的会有黑边，setInputTransparentColor后黑边没了，但是会导致图层发白，清晰度丢失，暂无解决办法，或许可以通过设置样式来改善，但是时间有限就没有试过了
