paddlepaddle--paddlehub--chinese

2023-09-24 本文已影响0人前浪浪奔浪流

第1步：安装Anaconda

说明：使用paddlepaddle需要先安装python环境，这里我们选择python集成环境Anaconda工具包
- Anaconda是1个常用的python包管理程序；
- 安装完Anaconda后，可以安装python环境，以及numpy等所需的工具包环境；
下载Anaconda：
下载地址：https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/?C=M&O=D

image.png
选择适合您操作系统的版本
- 可在终端输入uname -m查询系统所用的指令集

[root@test-web2 /]# uname -m
x86_64

本地下载，再将安装包传到linux服务器上
https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2023.07-2-Linux-x86_64.sh
- 若您要下载其他版本，需要将最后1个/后的文件名改成您希望下载的版本

- 安装Anaconda：
[root@test-web2 ~]# chmod +x Anaconda3-2023.07-2-Linux-x86_64.sh 
[root@test-web2 ~]# ll Anaconda3-2023.07-2-Linux-x86_64.sh 
-rwxr-xr-x 1 root root 1064920017 9月  25 11:17 Anaconda3-2023.07-2-Linux-x86_64.sh
sh Anaconda3-2023.07-2-Linux-x86_64.sh
若您下载的是其它版本，则将该命令的文件名替换为您下载的文件名
按照安装提示安装即可
查看许可时可输入q来退出

image.png

将conda加入环境变量

加入环境变量是为了让系统能识别conda命令，若您在安装时已将conda加入环境变量path，则可跳过本步
在终端中打开/etc/profile：

# 在终端中输入以下命令：
vim /etc/profile
... ...
export PATH="/root/anaconda3/bin:$PATH"

验证是否能识别conda命令：

source /etc/profile
conda info --envs   #若能显示当前有base环境，则conda已加入环境变量

image.png

第2步：创建conda环境

创建新的conda环境

# 在命令行输入以下命令，创建名为paddle_env的环境
# 此处为加速下载，使用清华源
conda create --name paddle_env python=3.10 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/

image.png

该命令会创建1个名为paddle_env、python版本为3.10的可执行环境，根据网络状态，需要花费一段时间
之后命令行中会输出提示信息，输入y并回车继续安装

激活刚创建的conda环境，在命令行中输入以下命令：

# 激活paddle_env环境
conda activate paddle_env

报错：

# conda activate paddle_env
usage: conda [-h] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'skeleton', 'pack', 'verify', 'token', 'repo', 'env', 'server', 'content-trust')

解决方法
查询当前环境位置，试用 source activate 进行激活就可以解决问题。

#查询环境位置
conda info | grep -i 'base environment' 
base environment : /root/anaconda3  (writable)
# /root/anaconda3 是你自己的输出结果
source /root/anaconda3/etc/profile.d/conda.sh 
#激活sh
conda activate paddle_env 
#可以运行了
conda deactivate
#退出虚拟环境
conda env remove -n paddle_env
#删除虚拟环境paddle_env

image.png

以上anaconda环境和python环境安装完毕

第3步：安装程序所需要库

使用pip命令在刚激活的环境中安装paddle：

# 在命令行中输入以下命令：
# 默认安装CPU版本，安装paddle时建议使用百度源
pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple

(paddle_env) [root@test-web2 ~]# pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
... ...
Installing collected packages: sentencepiece, pytz, pydub, paddle2onnx, jieba, ffmpy, easydict, xxhash, websockets, urllib3, tzdata, typing-extensions, tqdm, toolz, threadpoolctl, six, semantic-version, scipy, safetensors, rpds-py, rarfile, pyzmq, pyyaml, python-multipart, pyparsing, pygments, pycryptodome, pyarrow, psutil, protobuf, packaging, orjson, opencv-python, multidict, mdurl, markupsafe, kiwisolver, joblib, itsdangerous, importlib-resources, future, fsspec, frozenlist, fonttools, filelock, dill, cycler, contourpy, colorlog, colorama, click, charset-normalizer, blinker, Babel, attrs, async-timeout, anyio, annotated-types, aiofiles, yarl, Werkzeug, uvicorn, typer, starlette, scikit-learn, requests, referencing, python-dateutil, pydantic-core, onnx, multiprocess, markdown-it-py, Jinja2, gunicorn, bce-python-sdk, aiosignal, seqeval, rich, pydantic, pandas, paddlefsl, matplotlib, jsonschema-specifications, huggingface-hub, flask, aiohttp, jsonschema, gradio-client, Flask-Babel, fastapi, visualdl, datasets, altair, paddlenlp, gradio, paddlehub
  Attempting uninstall: protobuf
    Found existing installation: protobuf 4.24.3
    Uninstalling protobuf-4.24.3:
      Successfully uninstalled protobuf-4.24.3
  Attempting uninstall: anyio
    Found existing installation: anyio 4.0.0
    Uninstalling anyio-4.0.0:
      Successfully uninstalled anyio-4.0.0
Successfully installed Babel-2.12.1 Flask-Babel-3.1.0 Jinja2-3.1.2 Werkzeug-2.3.7 aiofiles-23.2.1 aiohttp-3.8.5 aiosignal-1.3.1 altair-5.1.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.3 attrs-23.1.0 bce-python-sdk-0.8.90 blinker-1.6.2 charset-normalizer-3.2.0 click-8.1.7 colorama-0.4.6 colorlog-6.7.0 contourpy-1.1.1 cycler-0.11.0 datasets-2.14.5 dill-0.3.4 easydict-1.10 fastapi-0.103.1 ffmpy-0.3.1 filelock-3.12.4 flask-2.3.3 fonttools-4.42.1 frozenlist-1.4.0 fsspec-2023.6.0 future-0.18.3 gradio-3.45.0 gradio-client-0.5.2 gunicorn-21.2.0 huggingface-hub-0.17.3 importlib-resources-6.1.0 itsdangerous-2.1.2 jieba-0.42.1 joblib-1.3.2 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 kiwisolver-1.4.5 markdown-it-py-3.0.0 markupsafe-2.1.3 matplotlib-3.8.0 mdurl-0.1.1 multidict-6.0.4 multiprocess-0.70.12.2 onnx-1.14.1 opencv-python-4.8.0.76 orjson-3.9.7 packaging-23.1 paddle2onnx-1.0.9 paddlefsl-1.1.0 paddlehub-2.4.0 paddlenlp-2.6.1 pandas-2.1.1 protobuf-3.20.2 psutil-5.9.5 pyarrow-13.0.0 pycryptodome-3.19.0 pydantic-2.4.1 pydantic-core-2.10.1 pydub-0.25.1 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3.post1 pyyaml-6.0.1 pyzmq-25.1.1 rarfile-4.1 referencing-0.30.2 requests-2.31.0 rich-13.5.3 rpds-py-0.10.3 safetensors-0.3.3.post1 scikit-learn-1.3.1 scipy-1.11.2 semantic-version-2.10.0 sentencepiece-0.1.99 seqeval-1.2.2 six-1.16.0 starlette-0.27.0 threadpoolctl-3.2.0 toolz-0.12.0 tqdm-4.66.1 typer-0.9.0 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.0.5 uvicorn-0.23.2 visualdl-2.5.3 websockets-11.0.3 xxhash-3.3.0 yarl-1.9.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
(paddle_env) [root@test-web2 ~]#

paddlepaddle-2.5.1

安装完paddle后，继续在paddle_env环境中安装paddlehub：

# 在命令行中输入以下命令：
pip install paddlehub -i https://mirror.baidu.com/pypi/simple
(paddle_env) [root@test-web2 ~]# pip install paddlehub -i https://mirror.baidu.com/pypi/simple
... ...
Successfully installed Babel-2.12.1 Flask-Babel-3.1.0 Jinja2-3.1.2 Werkzeug-2.3.7 aiofiles-23.2.1 aiohttp-3.8.5 aiosignal-1.3.1 altair-5.1.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.3 attrs-23.1.0 bce-python-sdk-0.8.90 blinker-1.6.2 charset-normalizer-3.2.0 click-8.1.7 colorama-0.4.6 colorlog-6.7.0 contourpy-1.1.1 cycler-0.11.0 datasets-2.14.5 dill-0.3.4 easydict-1.10 fastapi-0.103.1 ffmpy-0.3.1 filelock-3.12.4 flask-2.3.3 fonttools-4.42.1 frozenlist-1.4.0 fsspec-2023.6.0 future-0.18.3 gradio-3.44.4 gradio-client-0.5.1 gunicorn-21.2.0 huggingface-hub-0.17.2 importlib-resources-6.1.0 itsdangerous-2.1.2 jieba-0.42.1 joblib-1.3.2 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 kiwisolver-1.4.5 markdown-it-py-3.0.0 markupsafe-2.1.3 matplotlib-3.8.0 mdurl-0.1.1 multidict-6.0.4 multiprocess-0.70.12.2 onnx-1.14.1 opencv-python-4.8.0.76 orjson-3.9.7 packaging-23.1 paddle2onnx-1.0.9 paddlefsl-1.1.0 paddlehub-2.4.0 paddlenlp-2.6.1 pandas-2.1.1 protobuf-3.20.2 psutil-5.9.5 pyarrow-13.0.0 pycryptodome-3.19.0 pydantic-2.3.0 pydantic-core-2.6.3 pydub-0.25.1 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3.post1 pyyaml-6.0.1 pyzmq-25.1.1 rarfile-4.1 referencing-0.30.2 requests-2.31.0 rich-13.5.3 rpds-py-0.10.3 safetensors-0.3.3.post1 scikit-learn-1.3.1 scipy-1.11.2 semantic-version-2.10.0 sentencepiece-0.1.99 seqeval-1.2.2 six-1.16.0 starlette-0.27.0 threadpoolctl-3.2.0 toolz-0.12.0 tqdm-4.66.1 typer-0.9.0 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.0.5 uvicorn-0.23.2 visualdl-2.5.3 websockets-11.0.3 xxhash-3.3.0 yarl-1.9.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
(paddle_env) [root@test-web2 ~]#

安装paddlehub时会自动安装其它依赖库，可能需要花费一段时间
paddlehub-2.4.0

chinese_ocr_db_crnn_mobile模型依赖：

paddlepaddle >= 1.7.2
paddlehub >= 1.6.0
shapely
pyclipper

该Module依赖于第三方库shapely和pyclipper，使用该Module之前，请先安装shapely和pyclipper。

(paddle_env) [root@test-web2 ~]# pip install shapely pyclipper -i https://mirror.baidu.com/pypi/simple
Looking in indexes: https://mirror.baidu.com/pypi/simple
Collecting shapely
  Downloading https://mirror.baidu.com/pypi/packages/a8/a5/403728b5614b28083f6424dfdefec5fcf58068495fb03bb08532671c642f/shapely-2.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 7.6 MB/s eta 0:00:00
Collecting pyclipper
  Downloading https://mirror.baidu.com/pypi/packages/1c/81/4aa8403e587a4c60e00b479c11254a6e3200f3b985dcf4caecf0d8c21261/pyclipper-1.3.0.post5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (908 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 908.3/908.3 kB 4.4 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.14 in ./anaconda3/envs/paddle_env/lib/python3.10/site-packages (from shapely) (1.26.0)
Installing collected packages: pyclipper, shapely
Successfully installed pyclipper-1.3.0.post5 shapely-2.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

安装chinese_ocr_db_crnn_mobile模型

(paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple

报错：

(paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple
Error: Can not import paddle core while this file exists: /root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/libpaddle.so
Traceback (most recent call last):
  File "/root/anaconda3/envs/paddle_env/bin/hub", line 5, in <module>
    from paddlehub.commands.utils import execute
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddlehub/__init__.py", line 18, in <module>
    import paddle
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/__init__.py", line 31, in <module>
    from .framework import monkey_patch_variable
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/framework/__init__.py", line 17, in <module>
    from . import random  # noqa: F401
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/framework/random.py", line 17, in <module>
    from paddle import fluid
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/__init__.py", line 36, in <module>
    from . import framework
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/framework.py", line 35, in <module>
    from . import core
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/core.py", line 356, in <module>
    raise e
  File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/core.py", line 269, in <module>
    from . import libpaddle
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory

解决办法：
Centos7 默认提供的 openssl 版本是 1.0.2 的，如果过低的话就会报错 libssl.so.1.1: cannot open shared object file: No such file or directory，这是由于 openssl 库的位置不正确或未安装 1.1 以上版本的问题；

想要升级 openssl 版本则需要手动进行编译，报错解决方式很简单，安装正确即可

#下载 openssl
wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
或者
wget https://mirrors.cloud.tencent.com/openssl/source/openssl-1.1.1g.tar.gz
#编译 openssl
tar -xvf openssl-1.1.1g.tar.gz
cd openssl-1.1.1g
./config shared --openssldir=/usr/local/openssl --prefix=/usr/local/openssl
make && make install

编译完成后，使用 openssl version 来查看一下当前 openssl 版本号时，你会发现还是 1.0.2，所以这里需要做一些额外的配置工作

#配置
mv /usr/bin/openssl  /usr/bin/openssl.old
ln -s /usr/local/openssl/bin/openssl /usr/bin/openssl
[root@test-web2 ~]# find / -name libcrypto.so.1.1
/root/openssl-1.1.1g/libcrypto.so.1.1
/usr/local/openssl/lib/libcrypto.so.1.1

[root@test-web2 ~]# find / -name libssl.so.1.1
/root/openssl-1.1.1g/libssl.so.1.1
/usr/local/openssl/lib/libssl.so.1.1
[root@test-web2 ~]# ln -s /usr/local/openssl/lib/libssl.so.1.1 /usr/lib64/libssl.so.1.1
[root@test-web2 ~]# ln -s /usr/local/openssl/lib/libcrypto.so.1.1  /usr/lib64/libcrypto.so.1.1

ldconfig -v
#退出paddle_env虚拟环境
conda deactivate
#重新激活paddle_env
conda activate paddle_env
#查看openssl版本
(base) [root@test-web2 ~]# conda info --envs
# conda environments:
#
base                  *  /root/anaconda3

(base) [root@test-web2 ~]# openssl version
OpenSSL 1.1.1u  30 May 2023

(paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple
/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
usage: hub install [-h] [--ignore_env_mismatch]
hub install: error: unrecognized arguments: -i
(paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile
/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Download https://bj.bcebos.com/paddlehub/paddlehub_dev/chinese_ocr_db_crnn_mobile_1.2.0.zip
[##################################################] 100.00%
Decompress /root/.paddlehub/tmp/tmpphgh428g/chinese_ocr_db_crnn_mobile_1.2.0.zip
[##################################################] 100.00%
[2023-09-27 11:43:36,803] [    INFO] - Successfully installed chinese_ocr_db_crnn_mobile-1.2.0

(paddle_env) [root@test-web2 ~]# hub serving start -m chinese_ocr_db_crnn_mobile
/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
 * Serving Flask app 'paddlehub.serving.app_compat'
 * Debug mode: off
[2023-09-27 11:46:09,970] [    INFO] _internal.py:187 - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:8866
 * Running on http://192.168.100.251:8866
[2023-09-27 11:46:09,970] [    INFO] _internal.py:187 - Press CTRL+C to quit


# 直接启动
hub serving start -m chinese_ocr_db_crnn_server -p 8866 --use_multiprocess --workers=3
# 后台启动
nohup hub serving start -m chinese_ocr_db_crnn_server -p 8866 --use_multiprocess --workers=3 &
#--use_multiprocess 代表启动多线程
#--workers 代表线程数 一般为2*cpu-1
#关闭服务
hub serving stop -p 8866

chinese_ocr_db_crnn_mobile-1.2.0

另外：

ImportError: libSM.so.6: cannot open shared object file: No such file or directory
ImportError: libXrender.so.1: cannot open shared object file: No such file or directory
原因：
报错的主要原因是：缺少共享库；
解决办法
使用如下命令查看缺少的共享库

(base) [root@test-web2 ~]# yum whatprovides libSM.so.6
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * elrepo: mirrors.tuna.tsinghua.edu.cn
 * epel: mirrors.bfsu.edu.cn
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
libSM-1.2.2-2.el7.i686 : X.Org X11 SM runtime library
源    ：base
匹配来源：
提供    ：libSM.so.6

yum install libSM-1.2.2-2.el7
已加载插件：fastestmirror, langpacks
base                                                                                                                                  | 3.6 kB  00:00:00     
elrepo                                                                                                                                | 3.0 kB  00:00:00     
epel/x86_64/metalink                                                                                                                  | 6.1 kB  00:00:00     
epel                                                                                                                                  | 4.7 kB  00:00:00     
extras                                                                                                                                | 2.9 kB  00:00:00     
updates                                                                                                                               | 2.9 kB  00:00:00     
elrepo/primary_db                                                                                                                     | 357 kB  00:00:00     
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * elrepo: mirrors.tuna.tsinghua.edu.cn
 * epel: mirrors.bfsu.edu.cn
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
软件包 libSM-1.2.2-2.el7.x86_64 已安装并且是最新版本
如果发现不能安装 则使用下边的语句
yum install libSM-1.2.2-2.el7.x86_64 --setopt=protected_multilib=false

可能遇到的问题的参考办法：
https://www.cnblogs.com/newgold/p/17340518.html
详细的安装文档
https://blog.csdn.net/weixin_38997187/article/details/121680809
https://blog.csdn.net/sika_deer/article/details/120631075

paddlepaddle--paddlehub--chinese

第1步：安装Anaconda

下载Anaconda：

将conda加入环境变量

第2步：创建conda环境

激活刚创建的conda环境，在命令行中输入以下命令：

报错：

第3步：安装程序所需要库

chinese_ocr_db_crnn_mobile模型依赖：

安装chinese_ocr_db_crnn_mobile模型

另外：

猜你喜欢

热点阅读