EII(Edge Insights for Industrial

2021-10-13  本文已影响0人  Mr_Michael

一、简介

Edge Insights for Industrial

EII Documentation

github

Edge Insights for Industrial 是一组预先验证的成分，用于在边缘计算节点上集成视频和时间序列数据分析。

它有助于解决各种工业和制造用途，包括在整个工厂车间的各种硬件节点上进行数据收集、存储和分析。

1.软件架构

image

2.EII中的服务

Common EII services

  1. EtcdUI
  2. InfluxDBConnector
  3. OpcuaExport - Optional service to read from VideoAnalytics container to publish data to opcua clients
  4. RestDataExport - Optional service to read the metadata and image blob from InfluxDBConnector and ImageStore services respectively

Video related services

  1. VideoIngestion
  2. VideoAnalytics
  3. Visualizer
  4. WebVisualizer
  5. ImageStore
  6. AzureBridge
  7. FactoryControlApp - Optional service to read from VideoAnalytics container if one wants to control the light based on defective/non-defective data

Timeseries related services

  1. Telegraf
  2. Kapacitor
  3. Grafana
  4. ZMQ Broker

3.其他功能组件

1）TLS Server

Training and Learning Suite 2.0 (TLS) 使数据工程师只需点击几下即可轻松注释图像并创建 AI 模型。

2）TLS Remote Agent

是EII中的通信代理，与Training and Learning Suite（TLS）服务器交互，是EII的另一个模块应用，通过Web用户界面提供深度学习模型训练能力。TLS 可以设置为深度学习训练服务器，可以在单独的系统中或与 EII 相同的本地系统中。TLS 和 EII 都使用 mqtt 协议通过 RabbitMQ 代理进行通信。

二、安装

1.系统要求

2.通过Intel官网下载安装

镜像免编译，从官方hub下载

1）下载安装包

2）安装EII

3）EII配置文件安装路径

$ tree  /opt/intel/eii/
├── data
│   ├── etcd
│   │   ├── data
│   │   ├── fixtures
│   │   │   └── peer
│   │   │       ├── cert.pem
│   │   │       └── key.pem
│   │   └── member
│   │       ├── snap
│   │       │   └── db
│   │       └── wal
│   │           ├── 0000000000000000-0000000000000000.wal
│   │           └── 0.tmp
│   ├── image-store-bucket  # 存储持久化图像
│   └── influxdata  #  influxdb 的备份数据
│       └── influxdb
│           ├── data
│           │   ├── datain
│           │   │   ├── autogen
│           │   │   └── _series
│           │   └── _internal
│           │       ├── monitor
│           │       └── _series
│           ├── meta
│           │   └── meta.db
│           └── wal
│               ├── datain
│               │   └── autogen
│               └── _internal
│                   └── monitor
├── model_repo
├── rde_server_cert.der
├── rde_server_key.der
├── saved_images
├── sockets     # 存储 IPC ZMQ 套接字文件
│   └── camera1_stream
└── tools_output

4）edgesoftware工具

EII 服务安装管理工具

Usage: edgesoftware [OPTIONS] COMMAND [ARGS]...
A CLI wrapper for management of Intel® Edge Software Hub packages

Options:
  -v, --version   Show the version number and exit.
  ––help          Show this message and exit.

Commands:
  download  Download modules of a package.
  export    Exports the modules installed as a part of a package.
  install   Install modules of a package.
  list      List the modules of a package.
  log       Show log of CLI events.
  pull      Pull Docker image.
  uninstall Uninstall the modules of a package.
  update    Update the modules of a package.
  upgrade   Upgrade a package.

使用示例

$ cd edge_insights_industrial/
# 列举可以下载的服务
$ ./edgesoftware list --default
Modules in the recommended configuration for 60c862aeb3d614002abd9c6a
+--------------------------+------------------------------------------+---------+
|            ID            |                  Module                  | Version |
+--------------------------+------------------------------------------+---------+
| 5f21392e9e63c9002a6fd88d |      Docker Community Edition (CE)*      | 20.10.5 |
| 60e327614c1e9d002a6d6a7a |             Docker Compose*              |  1.29.0 |
| 60c7274a4b40e5002ad9b793 |             Video Analytics              |   2.6   |
| 60c8502c4b40e5002ad9b7bf |               Image Store                |   2.6   |
| 60c739334b40e5002ad9b7a0 |                InfluxDB*                 |   2.6   |
| 60c854824b40e5002ad9b7c6 |            Video Custom UDFs             |   2.6   |
| 60c73a174b40e5002ad9b7a8 |    Sample Factory Control Application    |   2.6   |
| 60c728244b40e5002ad9b79c |          Time Series Analytics           |   2.6   |
| 60caceef3c2973002abe2b3b | Intel® Training and Learning Suite (TLS) |   2.6   |
| 60c73ae14b40e5002ad9b7ac |                ZMQ Broker                |   2.6   |
| 60caceb13c2973002abe2b38 |           Device Manageability           |   2.6   |
| 60c73b324b40e5002ad9b7af |             Azure* IOT Edge              |   2.6   |
| 60c727f14b40e5002ad9b799 |               Data Export                |   2.6   |
| 60c726844b40e5002ad9b78b |                 EII Core                 |   2.6   |
| 60c850574b40e5002ad9b7c0 |           Video-Web-Visualizer           |   2.6   |
| 60c73a864b40e5002ad9b7ab |               Rest Export                |   2.6   |
| 60c739934b40e5002ad9b7a7 |              OPC-UA Export               |   2.6   |
| 60c7385a4b40e5002ad9b79d |                ts-grafana                |   2.6   |
| 60c738834b40e5002ad9b79e |               ts-kapacitor               |   2.6   |
| 60c738d94b40e5002ad9b79f |               ts-telegraf                |   2.6   |
| 60c846a94b40e5002ad9b7ba |             TLS Remote Agent             |   2.6   |
| 60c847e24b40e5002ad9b7bc |             Video-Ingestion              |   2.6   |
| 60c8507e4b40e5002ad9b7c3 |         Video-Native-Visualizer          |   2.6   |
| 60c8549b4b40e5002ad9b7c9 |             Video-Analytics              |   2.6   |
| 60c854ba4b40e5002ad9b7cc |               Video Common               |   2.6   |
| 60c727184b40e5002ad9b791 |               EII Samples                |   2.6   |
| 60c85f764b40e5002ad9b7d1 |                EII-Tools                 |   2.6   |
| 60c726ee4b40e5002ad9b78f |              EII-MessageBus              |   2.6   |
| 60c726c84b40e5002ad9b78d |               EII-C-Utils                |   2.6   |
| 60dae3b34472c4002a91d2d8 |               EII ETCD UI                |   2.6   |
| 60c727ad4b40e5002ad9b795 |              EII Installer               |   2.6   |
| 60e3270c4c1e9d002a6d6a78 |         Weld Porosity Detection          |   2.6   |
+--------------------------+------------------------------------------+---------+

# 卸载所有服务
$ ./edgesoftware uninstall -a

# 根据根据config_install.yml安装服务
$ ./edgesoftware install

3.通过git下载【推荐】

无需密钥，开源且商用免费，镜像需编译。

安装 repo 工具

$ curl https://storage.googleapis.com/git-repo-downloads/repo > repo
$ sudo mv repo /bin/repo
$ sudo chmod a+x /bin/repo

Pulling in the eii-manifests repo and using a manifest file

mkdir -p eii-manifests && cd eii-manifests
repo init -u "https://github.com/open-edge-insights/eii-manifests.git"
    # 将在当前目录下创建一个 .repo 文件夹，其中包含eii-manifests源代码.repo/manifests。

4.EII工程文件夹说明

~/Downloads/eii-manifests$ tree -L 3
.
└── IEdgeInsights
    ├── AzureBridge     # EII 服务
    │   ├── config
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── LICENSE
    │   ├── modules
    │   ├── README.md
    │   └── tools
    ├── build       # 镜像编译
    │   ├── ansible
    │   ├── builder_config.json
    │   ├── builder.py
    │   ├── builder_schema.json
    │   ├── common_config.json
    │   ├── common-docker-compose.yml
    │   ├── deploy
    │   ├── docker_security_recommendation.md
    │   ├── eii_uninstaller.sh
    │   ├── helm-eii
    │   ├── pre_requisites.sh
    │   ├── provision
    │   ├── remote_logging
    │   ├── requirements.txt
    │   └── usecases    # 根据不同场景定义的镜像集合
    ├── common
    │   ├── cmake
    │   ├── dockerfiles
    │   ├── eii_libs_installer.sh
    │   ├── libs
    │   ├── README.md
    │   ├── util
    │   └── video
    │       ├── config.json
    │       ├── docker-compose.yml
    │       ├── Dockerfile.openvino
    │       ├── Dockerfile.videocommon
    │       ├── install_pip_requirements.py
    │       ├── LICENSE
    │       ├── UDFLoader
    │       └── udfs    # 用户定义函数
    ├── CONTRIBUTING.md
    ├── CustomUdfs  # 单独容器化的Udf
    │   ├── GVASafetyGearIngestion
    │   ├── LICENSE
    │   ├── NativePclIngestion
    │   ├── NativeSafetyGearAnalytics
    │   ├── NativeSafetyGearIngestion
    │   ├── PyMultiClassificationIngestion
    │   ├── PySafetyGearAnalytics
    │   ├── PySafetyGearIngestion
    │   └── README.md
    ├── Etcd_Secrets_Configuration.md
    ├── EtcdUI
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── eii_nginx_dev.conf
    │   ├── eii_nginx_prod.conf
    │   ├── etcdkeeper
    │   ├── helm
    │   ├── img
    │   ├── nginx.conf
    │   ├── README.md
    │   ├── schema.json
    │   └── start_etcdkeeper.py
    ├── FactoryControlApp
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── factoryctrl_app.py
    │   ├── helm
    │   ├── HW_Configuration.pdf
    │   ├── __init__.py
    │   ├── LICENSE
    │   ├── README.md
    │   ├── requirements.txt
    │   └── schema.json
    ├── ImageStore
    │   ├── common
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── go
    │   ├── helm
    │   ├── isconfigmgr
    │   ├── LICENSE
    │   ├── main.go
    │   ├── README.md
    │   ├── schema.json
    │   ├── submanager
    │   └── test
    ├── InfluxDBConnector
    │   ├── common
    │   ├── config
    │   ├── config.json
    │   ├── configmanager
    │   ├── dbmanager
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── helm
    │   ├── InfluxDBConnector.go
    │   ├── influx_start.sh
    │   ├── LICENSE
    │   ├── pubmanager
    │   ├── README.md
    │   ├── schema.json
    │   ├── startup.sh
    │   ├── submanager
    │   └── test
    ├── Installing_docker_pre_requisites.md
    ├── LICENSE
    ├── licenses
    │   ├── ......
    │   └── wjelement
    ├── OpcuaExport
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── helm
    │   ├── LICENSE
    │   ├── OpcuaBusAbstraction
    │   ├── OpcuaExport.go
    │   ├── README.md
    │   └── schema.json
    ├── README.md
    ├── RestDataExport
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── etcd_update.py
    │   ├── helm
    │   ├── LICENSE
    │   ├── README.md
    │   ├── requirements.txt
    │   ├── RestDataExport.go
    │   └── schema.json
    ├── Samples
    │   ├── cpp_sample_app
    │   ├── go_sample_app
    │   ├── LICENSE
    │   ├── python_sample_app
    │   └── README.md
    ├── tools
    │   ├── Benchmarking
    │   ├── DiscoverHistory
    │   ├── EmbPublisher
    │   ├── EmbSubscriber
    │   ├── GigEConfig
    │   ├── HttpTestServer
    │   ├── JupyterNotebook
    │   ├── LICENSE
    │   ├── mqtt
    │   ├── SWTriggerUtility
    │   ├── TimeSeriesProfiler
    │   └── VideoProfiler
    ├── TROUBLESHOOT.md
    ├── VideoAnalytics
    │   ├── CMakeLists.txt
    │   ├── config.json
    │   ├── docker-compose-dev.override.yml
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── helm
    │   ├── include
    │   ├── LICENSE
    │   ├── README.md
    │   ├── schema.json
    │   ├── src
    │   └── va_classifier_start.sh
    ├── VideoIngestion
    │   ├── CMakeLists.txt
    │   ├── config.json
    │   ├── docker-compose-dev.override.yml
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── docs
    │   ├── gentl_producer_env.sh
    │   ├── helm
    │   ├── img
    │   ├── include
    │   ├── install_gencamsrc_gstreamer_plugin.sh
    │   ├── LICENSE
    │   ├── models
    │   ├── README.md
    │   ├── schema.json
    │   ├── src
    │   ├── src-gst-gencamsrc
    │   ├── test_videos
    │   └── vi_start.sh
    ├── Visualizer
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── helm
    │   ├── LICENSE
    │   ├── README.md
    │   ├── requirements.txt
    │   ├── schema.json
    │   ├── visualize.py
    │   └── visualizer_start.sh
    ├── WebVisualizer
    │   ├── config.json
    │   ├── docker-compose.yml
    │   ├── Dockerfile
    │   ├── helm
    │   ├── LICENSE
    │   ├── README.md
    │   ├── requirements.txt
    │   ├── schema.json
    │   ├── templates
    │   ├── web_visualizer.py
    │   └── web_visualizer_start.sh
    └── ZmqBroker
        ├── CMakeLists.txt
        ├── config.json
        ├── docker-compose.yml
        ├── Dockerfile
        ├── examples
        ├── helm
        ├── include
        ├── LICENSE
        ├── README.md
        ├── schema.json
        ├── src
        └── tests

5.编译安装与卸载EII服务（基于git）

安装依赖

cd eii-manifests/IEdgeInsights/build
sudo bash pre_requisites.sh

修改环境变量

cd eii-manifests/IEdgeInsights/build
vim .env
# 基于开发模式编译
    设置DEV_MODE=true
# 修改DOCKER_REGISTRY
    DOCKER_REGISTRY=registry.xxx:5000/
    DOCKER_USERNAME=
    DOCKER_PASSWORD=

设置环境变量并构建合并文件

cd eii-manifests/IEdgeInsights/build

# 使用 builder.py构建容器服务
python3 builder.py -h
usage: builder.py [-h] [-f YML_FILE] [-v VIDEO_PIPELINE_INSTANCES]
                  [-d OVERRIDE_DIRECTORY]
optional arguments:
  -h, --help            show this help message and exit
  -f YML_FILE, --yml_file YML_FILE
                        Optional config file for list of services to include.
                        Eg: python3 builder.py -f usecases/video-streaming.yml
                        (default: None)
  -v VIDEO_PIPELINE_INSTANCES, --video_pipeline_instances VIDEO_PIPELINE_INSTANCES
                        Optional number of video pipeline instances to be
                        created. Eg: python3 builder.py -v 6 (default: 1)
  -d OVERRIDE_DIRECTORY, --override_directory OVERRIDE_DIRECTORY
                        Optional directory consisting of of benchmarking
                        configs to be present ineach app directory. Eg:
                        python3 builder.py -d benchmarking (default: None)
                        
# 从IEdgeInsights顶层目录开始，轮询各级目录，结合所有EII服务生成合并文件
$ python3 builder.py 
$ ls docker-compose*
docker-compose-build.yml  docker-compose-push.yml  docker-compose.yml

    # 或者从IEdgeInsights顶层目录开始，根据不同应用场景，根据所选择的usecae文件包含的EII服务生成合并文件
    $ python3 builder.py -f usecases/video-streaming.yml

为EII服务生成证书和编译ia_etcd、ia_etcd_provision

cd provision && sudo -E ./provision.sh ../docker-compose.yml && cd -

编译与运行EII服务

cd eii-manifests/IEdgeInsights/build
# 通过EII各服务的Dockerfile构建镜像
docker-compose -f docker-compose-build.yml build
# 只构建某个镜像
docker-compose -f docker-compose-build.yml build ia_etcd_ui
# 启动容器
docker-compose up -d

docker-compose ps
2021-08-27 09-58-11 的屏幕截图.png

卸载EII服务

$ ./eii_uninstaller.sh -h
Usage: ./eii_uninstaller.sh [-h] [-d]

 This script uninstalls previous EII version.
 Where:
    -h show the help
    -d triggers the deletion of docker images (by default it will not trigger)

 Example:
  1) Deleting only EII Containers and Volumes
    $ ./eii_uninstaller.sh

  2) Deleting EII Containers, Volumes and Images
    $ export EII_VERSION=2.6
    $ ./eii_uninstaller.sh -d
    above example will delete EII containers, volumes and all the docker images having 2.4 version.

6.builder说明

builder是基于python开发的，可自动生成所需的配置文件，以在单/多节点上部署 EII 服务的工具。

1）builder关键文件

2）usecases

$ tree usecases/
usecases/
├── all.yml
├── services.yml
├── time-series.yml
├── video-streaming-all-udfs.yml
├── video-streaming-azure.yml
├── video-streaming-storage.yml
├── video-streaming.yml
├── video-timeseries.yml
└── video.yml

builder可以根据不同的use-cases构建各种服务

# 生成多实例配置：为 3 个视频流用例流生成多实例
$ python3 builder.py -v 3 -f usecases/video-streaming.yml

3）provision

Provision是EII基于安全模式的示例，用于生成EII服务的证书。

$ tree provision/ -L 2
provision/
├── cert_core.py
├── Certificates    # 证书文件
│   │   ├── AzureBridge
│   │   │   ├── AzureBridge_client_certificate.pem
│   │   │   └── AzureBridge_client_key.pem
│   │   ├── ca
│   │   │   ├── ca_certificate.der
│   │   │   ├── ca_certificate.pem
│   │   │   └── ca_key.pem
│   │   ├── etcdserver
│   │   │   ├── etcdserver_server_certificate.pem
│   │   │   └── etcdserver_server_key.pem
│   │   ├── EtcdUI
│   │   │   ├── EtcdUI_client_certificate.pem
│   │   │   └── EtcdUI_client_key.pem
│   │   ├── EtcdUI_Server
│   │   │   ├── EtcdUI_Server_server_certificate.pem
│   │   │   └── EtcdUI_Server_server_key.pem
│   │   ├── FactoryControlApp
│   │   │   ├── FactoryControlApp_client_certificate.pem
│   │   │   └── FactoryControlApp_client_key.pem
│   │   ├── ImageStore
│   │   │   ├── ImageStore_client_certificate.pem
│   │   │   └── ImageStore_client_key.pem
│   │   ├── InfluxDBConnector
│   │   │   ├── InfluxDBConnector_client_certificate.pem
│   │   │   └── InfluxDBConnector_client_key.pem
│   │   ├── InfluxDBConnector_Server
│   │   │   ├── InfluxDBConnector_Server_server_certificate.pem
│   │   │   └── InfluxDBConnector_Server_server_key.pem
│   │   ├── OpcuaExport
│   │   │   ├── OpcuaExport_client_certificate.pem
│   │   │   └── OpcuaExport_client_key.pem
│   │   ├── OpcuaExport_Server
│   │   │   ├── OpcuaExport_Server_server_certificate.der
│   │   │   └── OpcuaExport_Server_server_key.der
│   │   ├── RestDataExport
│   │   │   ├── RestDataExport_client_certificate.pem
│   │   │   └── RestDataExport_client_key.pem
│   │   ├── RestDataExport_Server
│   │   │   ├── RestDataExport_Server_server_certificate.pem
│   │   │   └── RestDataExport_Server_server_key.pem
│   │   ├── root
│   │   │   ├── root_client_certificate.pem
│   │   │   └── root_client_key.pem
│   │   ├── VideoAnalytics
│   │   │   ├── VideoAnalytics_client_certificate.pem
│   │   │   └── VideoAnalytics_client_key.pem
│   │   ├── VideoIngestion
│   │   │   ├── VideoIngestion_client_certificate.pem
│   │   │   └── VideoIngestion_client_key.pem
│   │   ├── Visualizer
│   │   │   ├── Visualizer_client_certificate.pem
│   │   │   └── Visualizer_client_key.pem
│   │   ├── WebVisualizer
│   │   │   ├── WebVisualizer_client_certificate.pem
│   │   │   └── WebVisualizer_client_key.pem
│   │   ├── WebVisualizer_Server
│   │   │   ├── WebVisualizer_Server_server_certificate.pem
│   │   │   └── WebVisualizer_Server_server_key.pem
│   │   └── ZmqBroker
│   │       ├── ZmqBroker_client_certificate.pem
│   │       └── ZmqBroker_client_key.pem
├── cert_requirements.txt
├── config
│   ├── eii_config.json     # EII配置示例
│   ├── openssl.cnf
│   └── x509_cert_config.json       #  X509 证书配置
├── dep
│   ├── docker-compose-etcd.override.build.yml
│   ├── docker-compose-etcd.override.prod.yml
│   ├── docker-compose-etcd-provision.override.build.yml
│   ├── docker-compose-etcd-provision.override.prod.yml
│   ├── docker-compose-etcd-provision.yml
│   ├── docker-compose-etcd.yml
│   ├── Dockerfile
│   ├── Dockerfile.provision
│   ├── etcd_config_update.py
│   ├── etcd_create_user.sh
│   ├── etcd_enable_auth.sh
│   ├── etcd_health_check.sh
│   ├── etcd_provision.py
│   ├── requirements.txt
│   ├── start_etcd.sh
│   └── util.py
├── etcd_capture.py
├── etcd_capture.sh
├── gen_certs.py
├── paths.py
├── provision.sh
└── rootca

4）helm-eii

用于启用GenICam GigE 相机。

三.EII核心服务

1.ETCD-UI

一旦 EII 配置管理 (ia_etcd) 服务成功启动，用户可以通过以下步骤访问 ETCD Web UI。这允许用户对相应的 EII 容器服务进行配置更改。

[站外图片上传中...(image-28682f-1634094655794)]

注意：

2.InfluxDBConnector

InfluxDBConnector 将订阅 InfluxDB 并根据 PubTopics、SubTopics 和 QueryTopics 配置启动 zmq 发布者线程、zmq 订阅者线程和 zmq 请求回复线程。

{
    "config": {
        "influxdb": {
            "retention": "1h30m5s",
            "username": "admin",
            "password": "admin123",
            "dbname": "datain",
            "ssl": "True",
            "verifySsl": "False",
            "port": "8086"
        },
        "pub_workers": "5",
        "sub_workers": "5",
        "ignore_keys": [ "defects" ],
        "tag_keys": [],
        "blacklist_query": ["CREATE","DROP","DELETE","ALTER","<script>"]
    },
    "interfaces": {
        "Servers": [
            {
                "Name": "InfluxDBConnector",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65145",
                "AllowedClients": [
                    "*"
                ]
            }
        ],
        "Publishers": [
            {
                "Name": "PointData",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65033",
                "Topics": [
                    "point_data"
                ],
                "AllowedClients": [
                    "*"
                ]
            },
            {
                "Name": "PointClassifierResults",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65034",
                "Topics": [
                    "point_classifier_results"
                ],
                "AllowedClients": [
                    "*"
                ]
            },
            {
                "Name": "HumidityClassifierResults",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65030",
                "Topics": [
                    "humidity_classifier_results"
                ],
                "AllowedClients": [
                    "*"
                ]
            },
            {
                "Name": "TSData",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65031",
                "Topics": [
                    "ts_data"
                ],
                "AllowedClients": [
                    "*"
                ]
            },
            {
                "Name": "RFCResults",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65032",
                "Topics": [
                    "rfc_results"
                ],
                "AllowedClients": [
                    "*"
                ]
            }
        ],
        "Subscribers": [
            {
                "Name": "video",
                "Type": "zmq_tcp",
                "EndPoint": "ia_video_analytics:65013",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ]
            }   
        ]
        }
}

3.RestDataExport

RestDataExport 服务从 EIIMessageBus 订阅任意主题，并开始通过 POST 请求向任何外部 HTTP 服务器发布元数据。它有一个内部 HTTP 服务器运行，以响应来自任何 HTTP 客户端的对所需帧的任何 GET 请求。

image

1）运行HttpTestServer并修改配置

RestDataExport默认并没有指定httpserver，因此需要运行一个httpserver，并且将订阅的主题指定到httpserver

$ cd IEdgeInsights/tools/HttpTestServer
# 基于本地批生产证书
$ ./generate_testserver_cert.sh 0.0.0.0
# 基于开发模式运行，无需密钥

$ go mod init HttpTestServer && go mod tidy
$ go run TestServer.go --dev_mode true --host=0.0.0.0 --port=8082 --rdehost=localhost --rdeport=8087

修改/RestDataExport/config

{
    "camera1_stream_results": "http://172.17.0.1:8082",
    "http_server_ca": "/opt/intel/eii/cert.pem",
    "point_classifier_results": "http://172.17.0.1:8082",
    "rest_export_server_host": "0.0.0.0",
    "rest_export_server_port": "8087"
}

数据交互

# ia_rest_export post request
ia_rest_export                     | I0828 02:54:19.587145       1 RestDataExport.go:414] Imghandle ef4519ab02 and md5sum [212 29 140 217 143 0 178 4 233 128 9 152 236 248 66 126]
ia_rest_export                     | I0828 02:54:19.588396       1 RestDataExport.go:308] Response : Received a POST request

# HttpTestServer接收数据
I0828 10:54:19.580769   10928 TestServer.go:241] Received metadata : map[channels:3 encoding_level:95 topic:camera1_stream_results width:1920 defects:[map[br:[1006 689] tl:[958 635] type:0] map[br:[1297 768] tl:[1242 733] type:0] map[br:[1315 913] tl:[1270 877] type:1]] encoding_type:jpeg frame_number:1306 height:1200 img_handle:ef4519ab02]
I0828 10:54:19.588081   10928 TestServer.go:189] imgHandle ef4519ab02 and md5sum [212 29 140 217 143 0 178 4 233 128 9 152 236 248 66 126]

# camera1_stream_results metadata
map[
    channels:3 
    encoding_level:95 
    topic:camera1_stream_results 
    width:1920 
    height:1200 
    defects:[
        map[br:[1006 689] tl:[958 635] type:0] 
        map[br:[1297 768] tl:[1242 733] type:0] 
        map[br:[1315 913] tl:[1270 877] type:1]] 
    encoding_type:jpeg 
    frame_number:1306 
    img_handle:ef4519ab02
]

2）RestExport API

RestExport API 源代码位于[WORK_DIR]/rest_export/RestExport.go 中。它负责读取 RestExport 配置并管理 RestExport 管道的生命周期。

3）示例config.json

{
    "config": {
        "camera1_stream_results": "http://<IP Address of Test Server>:8082",        // 目标http服务器
        "point_classifier_results": "http://<IP Address of Test Server>:8082",
        "http_server_ca": "/opt/intel/eii/cert.pem",
        "rest_export_server_host": "0.0.0.0",
        "rest_export_server_port": "8087"
    },
    "interfaces": {
        "Clients": [
            {
                "Name": "ImageStore",
                "ServerAppName": "ImageStore",
                "Type": "zmq_tcp",
                "EndPoint": "ia_imagestore:5669"
            }
        ],
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "ia_video_analytics:65013",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ]
            },
            {
                "Name": "influxSub",
                "Type": "zmq_tcp",
                "EndPoint": "ia_influxdbconnector:65034",
                "PublisherAppName": "InfluxDBConnector",
                "Topics": [
                    "point_classifier_results"
                ]
            }
        ]
    }
}

4.VideoIngestion

VideoIngestion(VI) 模块主要负责将来自视频文件或 basler/RTSP/USB 相机等视频源的视频帧摄取到 EII 堆栈中以进行进一步处理。

[站外图片上传中...(image-f3d525-1634094655795)]

1）Video Ingestion 工作流程

  1. 应用程序通过 EII 配置管理器读取应用程序配置，其中包含ingestor、encoding和 的详细信息udfs。
  2. 根据摄取器配置，应用程序从视频文件或摄像头读取视频帧。
  3. [ 可选 ] 读取帧被传递到一个或多个链接的C++/python UDF 以进行任何预处理（通过 UDF 传递是可选的，如果不想对摄取的帧执行任何预处理，则不需要） 。 通过支持 UDF 链接，还可以拥有分类器 UDF 和任何后处理 UDF，如调整大小等，在udfskey 中配置 以获得分类结果。
  4. 应用程序从系统环境中获取 msgbus 端点配置，并根据配置，应用程序在 EII MessageBus 上发布有关上述主题的数据。

2）Ingestors

Video file

RTSP Camera

USB Camera

3）encoding

4）示例config.json

{
    "config": {
        "encoding": {
            "type": "jpeg",
            "level": 95
        },
        "ingestor": {
            "type": "opencv",
            "pipeline": "./test_videos/pcb_d2000.avi",
            "loop_video": true,
            "queue_size": 10,
            "poll_interval": 0.2
        },
        "sw_trigger": {
            "init_state": "running"
        },
        "max_workers":4,
        "udfs": [{
            "name": "pcb.pcb_filter",
            "type": "python",
            "scale_ratio": 4,
            "training_mode": "false",
            "n_total_px": 300000,
            "n_left_px": 1000,
            "n_right_px": 1000
        }]
    },
    "interfaces": {
        "Servers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:64013",
                "AllowedClients": [
                    "*"
                ]
            }
        ],
        "Publishers": [
            {
                "Name": "default",
                "Type": "zmq_ipc",
                "EndPoint": "/EII/sockets",
                "Topics": [
                    "camera1_stream"
                ],
                "AllowedClients": [
                    "VideoAnalytics", "Visualizer", "WebVisualizer", "TLSRemoteAgent", "RestDataExport"
                ]
            }
        ]
    }
}

5.VideoAnalytics

VideoAnalytics 模块主要负责运行分类器 UDF 并使用 openVINO 在选定的 Intel(R) 硬件（CPU、GPU、VPU、HDDL）上进行所需的推理。

VideoAnalytics 的运作流程如下：

  1. 应用程序通过 EII 配置管理器读取应用程序配置，其中包含encoding和 的详细信息udfs。
  2. App 从系统环境中获取 msgbus 端点配置。
  3. 基于以上两种配置，应用程序订阅来自 VideoIngestion 模块的已发布主题/流。
  4. 订阅者收到的帧被传递到一个或多个链接的C++/python UDF 上，用于运行推理并根据需要进行任何后处理。
  5. 来自链式 udf 的帧发布在 EII MessageBus 上的不同主题/流上。

示例config.json

{
    "config": {
        "encoding": {
            "type": "jpeg",
            "level": 95
        },
        "queue_size": 10,
        "max_workers":4,
        "udfs": [{
            "name": "pcb.pcb_classifier",
            "type": "python",
            "ref_img": "common/video/udfs/python/pcb/ref/ref.png",
            "ref_config_roi": "common/video/udfs/python/pcb/ref/roi_2.json",
            "model_xml": "common/video/udfs/python/pcb/ref/model_2.xml",
            "model_bin": "common/video/udfs/python/pcb/ref/model_2.bin",
            "device": "CPU"
        }]
    },
    "interfaces": {
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_ipc",
                "EndPoint": "/EII/sockets",
                "PublisherAppName": "VideoIngestion",
                "Topics": [
                    "camera1_stream"
                ],
                "zmq_recv_hwm": 50
            }
        ],
        "Publishers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:65013",
                "Topics": [
                    "camera1_stream_results"
                ],
                "AllowedClients": [
                    "*"
                ]
            }
        ]
    }
}

6.Visualizer

video-native-visualizer是一个本地应用程序，用于在本地桌面查看来自 EII 的分类图像/元数据，需要连接显示屏。

示例config.json

{
    "config": {
        "save_image": "false",
        "draw_results": "true",
        "labels" : {
            "camera1_stream_results": {
                "0": "MISSING",
                "1": "SHORT"
            },
            "native_safety_gear_stream_results": {
                "1": "safety_helmet",
                "2": "safety_jacket",
                "3": "Safe",
                "4": "Violation"
            },
            "py_safety_gear_stream_results": {
                "1": "safety_helmet",
                "2": "safety_jacket",
                "3": "Safe",
                "4": "Violation"

            },
            "gva_safety_gear_stream_results": {
                "1": "safety_helmet",
                "2": "safety_jacket",
                "3": "Safe",
                "4": "Violation"
            }

        }
    },
    "interfaces": {
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "ia_video_analytics:65013",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ]
            }
        ]
    }
}

7.WebVisualizer

video-web-visualizer是一个基于 Web 的应用程序，用于在浏览器上查看来自 EII 的分类图像/元数据。

在浏览器中运行 Visualizer：

示例config.json

"/WebVisualizer/config": {
      "username": "admin",
      "password": "admin@123",
      "dev_port": 5001,
      "port": 5000,
      "labels" : {
          "camera1_stream": {
              "0": "MISSING",
              "1": "SHORT"
          },
          "native_safety_gear_stream_results": {
              "1": "safety_helmet",
              "2": "safety_jacket",
              "3": "Safe",
              "4": "Violation"
          },
          "py_safety_gear_stream_results": {
              "1": "safety_helmet",
              "2": "safety_jacket",
              "3": "Safe",
              "4": "Violation"

          },
          "gva_safety_gear_stream_results": {
              "1": "safety_helmet",
              "2": "safety_jacket",
              "3": "Safe",
              "4": "Violation"
          }

      }
  }

8.ImageStore

EII 的 Image Store 组件作为一个单独的容器出现，它主要通过 EII MessageBus 订阅来自 VideoAnalytics 应用程序的流，并将帧存储到 minio db 中以进行历史分析。

ImageStore 的工作流程如下：

  1. ImageStore 中的消息总线订阅者将订阅 VideoAnalytics 在消息总线上发布的分类结果（元数据、帧）。img_handle 是从元数据中提取出来的，用作键，帧作为该键的值存储在 minio 持久存储中。

    注意：测试发现订阅1路AI结果储存正常，订阅两路及以上无法正常储存图片。

  2. 为了对存储的分类图像进行历史分析，ImageStore 启动提供读取和存储接口的消息总线服务器。有效载荷格式如下：

    存储接口

    Request: map ("command": "store","img_handle":"$handle_name"),[]byte($binaryImage)
Response : map ("img_handle":"$handle_name", "error":"$error_msg") ("error" is optional and available only in case of error in execution.)

    ia_imagestore                      | I0828 08:03:47.403717       1 subManager.go:126] Image with handle f8befd3737 stored successfully
ia_imagestore                      | I0828 08:03:51.141884       1 main.go:268] Successfully read frame with handle:20bae8930f
ia_imagestore                      | I0828 08:03:51.151968       1 main.go:268] Successfully read frame with handle:f8befd3737
ia_imagestore                      | I0828 08:03:55.260218       1 subManager.go:111] 
ia_imagestore                      | -- Received Message: map[channels:3 defects:[] encoding_level:95 encoding_type:jpeg frame_number:32385 height:1200 img_handle:d92ef1a2bb width:1920]
ia_imagestore                      | I0828 08:03:55.260341       1 subManager.go:126] Image with handle d92ef1a2bb stored successfully
ia_imagestore                      | I0828 08:03:55.290998       1 main.go:268] Successfully read frame with handle:d92ef1a2bb

    读取接口

    Request : map ("command": "read", "img_handle":"$handle_name")
Response : map ("img_handle":"$handle_name", "error":"$error_msg"),[]byte($binaryImage)

    ("error" is optional and available only in case of error in execution. And $binaryImage is available only in case of successful read)

    可以通过RestDataExport取图

示例config.json

{
    "config": {
        "minio": {
            "accessKey": "admin",       // minio db所需的用户名
            "secretKey": "password",
            "retentionTime": "1h",
            "retentionPollInterval": "60s",
            "ssl": "false"
        }
    },
    "interfaces": {
        "Servers": [
            {
                "Name": "ImageStore",
                "Type": "zmq_tcp",
                "EndPoint": "0.0.0.0:5669",
                "AllowedClients": [
                    "*"
                ]
            }
        ],
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "ia_video_analytics:65013",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ]
            }
        ]
    }
}

9.ZMQ Broker

ZeroMQ Broker can be used in Publish-Subscribe as well as Request-Response modes.

传统上，每个 EII 消息总线 ZeroMQ 发布者都有自己的 IPC 套接字或 TCP（主机、端口）组合。此代理允许发布者连接到代理的发布者 TCP 或 IPC 套接字以发送已发布的消息和订阅者连接到一个中央订阅者 TCP 或 IPC 套接字以接收来自代理的传入消息。

ZeroMQ Broker 建立在 ZeroMQ zmq_proxy()API提供的功能之上 ，它使用两个独立的套接字并将消息从一个套接字传递到另一个套接字。

EII Message Bus是对 ZeroMQ* 的抽象，用于所有容器间通信。

1）ZeroMQ 代理运作流程

 +-----------+    +-----------+    +-----------+
 | Publisher |    | Publisher |    | Publisher |
 +-----------+    +-----------+    +-----------+
 |  ZMQ_PUB  |    |  ZMQ_PUB  |    |  ZMQ_PUB  |
 +-----------+    +-----------+    +-----------+
    connect          connect          connect
      |                |                 |
      +----------------+-----------------+
                       |
                      bind
               +----------------+
               |    ZMQ_XSUB    |       # 代理统一订阅端
               +----------------+
               |     proxy      |
               +----------------+
               |    ZMQ_XPUB    |       # 代理统一发布端
               +----------------+
                      bind
                       |
       +---------------+-------------------+
       |               |                   |
    connect          connect            connect
 +------------+    +------------+    +------------+
 |  ZMQ_SUB   |    |  ZMQ_SUB   |    |  ZMQ_SUB   |
 +------------+    +------------+    +------------+
 | Subscriber |    | Subscriber |    | Subscriber |
 +------------+    +------------+    +------------+

代理使用两个 ZeroMQ 套接字：

  1. 一个供发布者连接的ZMQ_XSUB socket ，简称前端socket
  2. 一个供订阅者连接的ZMQ_XPUB socket，称为后端socket

代理为其每个套接字绑定到 TCP（IP、端口）或 IPC 套接字文件。消息从发布者发送到前端套接字，然后中继到后端套接字，后者将消息转发给订阅者。

2）ZMQ_IPC

ZeroMQ 中的 IPC 通信模式使用 Linux* IPC 套接字，其速度比 TCP通信模式快。 IPC 模式不加密数据，它仅用于通信在同一节点上运行的应用程序之间以及需要传输大量以更快的速度传输数据（如视频帧）。视频帧通常非常大，加密它们会降低性能。

所有使用 IPC 通信方式的 EII 容器都在同一个 Linux 下执行用户组，所创建的套接字文件授予同一用户组的读取权限。因此，只有经过授权的 EII 应用程序才能访问和读取这些套接字文件中的数据。在 IPC 中的应用模式需要由管理员审查，因为他们可以访问以 IPC 模式发布的所有流。

"/VideoIngestion/interfaces": {
        "Publishers": [
            {
                "AllowedClients": [
                    "VideoAnalytics",
                    "Visualizer",
                    "WebVisualizer",
                    "TLSRemoteAgent",
                    "RestDataExport"
                ],
                "EndPoint": "/EII/sockets",
                "Name": "default",
                "Topics": [
                    "camera1_stream"
                ],
                "Type": "zmq_ipc"
            }
        ],
    }

"/VideoAnalytics/interfaces": {
        "Subscribers": [
            {
                "EndPoint": "/EII/sockets",
                "Name": "default",
                "PublisherAppName": "VideoIngestion",
                "Topics": [
                    "camera1_stream"
                ],
                "Type": "zmq_ipc",
                "zmq_recv_hwm": 50
            }
        ]
    },

3）ZMQ_TCP

在 TCP 模式下使用时，流的发布者将绑定到 TCP 套接字，之后订阅者连接到该套接字以接收数据。在请求-响应模式中，响应者充当服务器并绑定套接字，而请求者连接到它。

当我们需要将数据从生成数据的节点发送出去，将使用 TCP 模式。对于 TCP 模式，使用 CurveZMQ 协议启用全数据加密。

CurveZMQ 使用 CurveCP 握手机制，这与 mTLS 不同。 CurveZMQ不使用任何 CA；因此，客户端需要服务器的公钥，而服务器需要客户端的公钥以建立相互认证的连接。

Publish-Subscribe

分别作为Publisher和Subscriber

"/VideoAnalytics/interfaces": {
        "Publishers": [
            {
                "AllowedClients": [
                    "*"
                ],
                "EndPoint": "0.0.0.0:65013",
                "Name": "default",
                "Topics": [
                    "camera1_stream_results"
                ],
                "Type": "zmq_tcp"
            }
        ]
    }

"/ImageStore/interfaces": {
        "Subscribers": [
            {
                "EndPoint": "ia_video_analytics:65013",
                "Name": "default",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ],
                "Type": "zmq_tcp"
            }
        ]
    }

Request-Response

分别作为Server和Client

 "/ImageStore/interfaces": {
        "Servers": [
            {
                "AllowedClients": [
                    "*"
                ],
                "EndPoint": "0.0.0.0:5669",
                "Name": "ImageStore",
                "Type": "zmq_tcp"
            }
        ],
    }

"/RestDataExport/interfaces": {
        "Clients": [
            {
                "EndPoint": "ia_imagestore:5669",
                "Name": "ImageStore",
                "ServerAppName": "ImageStore",
                "Type": "zmq_tcp"
            }
        ],
    }

四、UDFs(video-common)

UDFs编写指导

UDF（User Defined Function）是 EII 框架的主要功能之一，它是一块用户代码，用于对EII输入数据进行过滤、预处理或分类等操作。EII支持加载和运行Native C++或python版本的UDFs API。

UDF 使用内存消息传递而不是套接字来进行管道和 UDF 之间的通信，因此相比之下速度更快。

UDF层级结构如下：

2021-08-24 14-21-42 的屏幕截图.png

1.UDF目录说明

IEdgeInsights/common/video# tree -L 2
.
├── config.json
├── docker-compose.yml
├── Dockerfile.openvino
├── Dockerfile.videocommon
├── install_pip_requirements.py
├── LICENSE
├── UDFLoader
│   ├── cmake
│   ├── CMakeLists.txt
│   ├── examples
│   ├── include
│   ├── README.md
│   ├── requirements.txt
│   ├── src
│   └── tests
└── udfs    # UDP Samples
    ├── HOWTO_GUIDE_FOR_WRITING_UDF.md
    ├── native
    │   ├── CMakeLists.txt
    │   ├── dummy
    │   │   ├── CMakeLists.txt
    │   │   └── dummy.cpp   # 真正对应的udf代码
    │   ├── fps
    │   │   ├── CMakeLists.txt
    │   │   ├── fps.cpp
    │   │   └── fps.h
    │   ├── raw_dummy
    │   │   ├── CMakeLists.txt
    │   │   └── raw_dummy.cpp
    │   ├── resize
    │   │   ├── CMakeLists.txt
    │   │   └── resize.cpp
    │   └── sample_realsense
    │       ├── CMakeLists.txt
    │       └── sample_realsense.cpp
    ├── python
    │   ├── dummy.py
    │   ├── jupyter_connector.py
    │   ├── multi_frame_dummy.py
    │   ├── pcb
    │   │   ├── __init__.py
    │   │   ├── pcb_classifier.py
    │   │   ├── pcb_filter.py
    │   │   ├── README.md
    │   │   └── ref
    │   └── sample_onnx
    │       ├── __init__.py
    │       ├── onnx_udf.py
    │       └── requirements.txt
    └── README.md

2.UDP Samples

目前官方给出的UDP示例，用户需要根据实际情况，编写自己的UDF程序

1）Native(C++) UDF

C++（它也被称为Native UDF，因为 EII 核心组件是用 C++ 实现的）

2）Python UDFs

3）Metadata结构

保元数据中的数据应该是列表、元组、字典或原始数据类型（int、float、string 或 bool）。此外，具有 in list、tuple、dict 的数据必须仅包含原始数据类型。例如：任何类型为“numpy.float”或“numpy.int”的数据都应分别转换为 float 和 int 类型。

3.C++ UDF的编写步骤

每个 UDF 编写都有两个主要部分。

1）EII Public API

这些 API 必须作为从BaseUdf（Udf 类）继承的用户定义类的方法来实现。

2）EII 基础设施变化

4.Python UDF 的编写步骤

编写 Python UDF 的过程，也有两个方面

1）EII Public Python API

返回值：此函数返回三个值。

2）EII 基础设施变化

5.UDF构建独立容器

UDF 可以在EII 构建环境的udfs-path（common/video/udfs）中创建，以便它可以编译到 VI（视频摄取）和 VA（视频分析）容器中。除此外，每个 UDF 都可以构建为基于 VI（VideoIngestion）或 VA（VideoAnalytics）容器镜像的独立容器。

独立容器的好处

示例CustomUdfs目录

# Edge_Insights_for_Industrial_2.6/IEdgeInsights
│       ├── CustomUdfs      # 自定义的UDF程序
│       │   ├── GVASafetyGearIngestion
│       │   ├── LICENSE
│       │   ├── NativePclIngestion
│       │   ├── NativeSafetyGearAnalytics
│       │   ├── NativeSafetyGearIngestion
│       │   ├── PyMultiClassificationIngestion
│       │   ├── PySafetyGearAnalytics
│       │   ├── PySafetyGearIngestion
│       │   └── README.md

1）UDF 容器目录布局

https://github.com/open-edge-insights/video-custom-udfs

Native C++ UDF 容器源示例

NativeSafetyGearAnalytics
├── config.json     # 定义了 UDF 特定配置和其他通用配置
├── docker-compose.yml
├── Dockerfile
└── safety_gear_demo
    ├── CMakeLists.txt
    ├── ref # AI模型
    │   ├── frozen_inference_graph.bin
    │   ├── frozen_inference_graph_fp16.bin
    │   ├── frozen_inference_graph_fp16.xml
    │   └── frozen_inference_graph.xml
    ├── safety_gear_demo.cpp
    └── safety_gear_demo.h

python UDF容器源示例

PyMultiClassificationIngestion
├── config.json
├── docker-compose.yml
├── Dockerfile
└── sample_classification
    ├── __init__.py
    ├── multi_class_classifier.py
    └── ref
        ├── squeezenet1.1_FP16.bin
        ├── squeezenet1.1_FP16.xml
        ├── squeezenet1.1_FP32.bin
        ├── squeezenet1.1_FP32.xml
        └── squeezenet1.1.labels

2）部署流程

上一篇下一篇

猜你喜欢

热点阅读