FastDFS+Nginx搭建文件存储服务器(单机版本)以及整合
2018-11-27 本文已影响0人
mr_xl
fastdfs的github地址: https://github.com/happyfish100/fastdfs
fastdfs的java客户端github地址: https://github.com/happyfish100/fastdfs-client-java(这个是官方的客户端) / https://github.com/tobato/FastDFS_Client(用这个客户端)
搭建环境 centos
一、准备工作
1.上传所需文件并解压
fastdfs-5.11.tar.gz
fastdfs-nginx-module-1.20.tar.gz
libfastcommon-1.0.39.tar.gz
nginx-1.14.1.tar.gz
2.安装libevent库
yum -y install libevent
4.进入libfastcommon-1.0.39目录中执行安装命令(安装后能在/usr/lib64或者/usr/lib目录中找到)
./make.sh && ./make.sh install
5.在上面两个目录中查找libfastcommon.so , 如果只在64目录中找到就把它复制到/usr/lib目录中
二、安装tracker
6.进入fasfdfs的解压目录下
cd fastdfs-5.11
7.执行编译和安装命令, 会安装在/usr/bin 和/etc/fdfs/目录下，使用 ll /usr/bin/fdfs* 查看
./make.sh && ./make.sh install
8.进入fastdfs5.11/conf目录中，把里面所有的配置文件复制到/etc/fdfs目录中
cp * /etc/fdfs
9.进入/etc/fdfs/目录 中使用命令打开
vim tracker.conf
修改basepath为 /fastdfs/tracker
到根目录下创建这些目录
mkdir /fastdfss/tracker -p
mkdir /fastdfs/storage -p
mkdir /fastdfs/client -p
mkdir /fastdfs/temp -p
10.使用命令和配置启动tracker
/usr/bin/fdfs_trackerd /etc/fdfs/tracker_conf
/usr/bin/fdfs_trackerd /etc/fdfs/tracker_conf restart 重启服务
三、安装storage
11.到/etc/fdfs目录下修改 storage.conf
basepath=/fasddfs/storage
store_path0=/fastdfs/storage
group_name=sl(这个组名自己定义)
tracker_server=192.168.213.120:22122（本机ip）
12.使用命令启动
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart 重启
13.查看状态
ps aux|grep tracker ps aux|grep storage
四、测试上传
14.到/etc/fdfs/目录下修改client.conf
basepath=/fastdfs/client
tracker_server=192.169.213.120:22122（本机）
15.使用下面的测试命令上传一张图片，会返回地址
命令：/usr/bin/fdfs_test /etc/fdfs/client.conf upload yusheng.jpg
返回
This is FastDFS client test program v5.11
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2018-11-20 04:36:38] DEBUG - base_path=/fastdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
tracker_query_storage_store_list_without_group:
server 1. group_name=, ip_addr=192.168.213.120, port=23000
group_name=sl, ip_addr=192.168.213.120, port=23000
storage_upload_by_filename
group_name=sl, remote_filename=M00/00/00/wKjVeFvz_9aAcMNOAAEJsJdjMjM460.jpg
source ip address: 192.168.213.120
file timestamp=2018-11-20 04:36:38
file size=68016
file crc32=2539860531
example file url: http://192.168.213.120/sl/M00/00/00/wKjVeFvz_9aAcMNOAAEJsJdjMjM460.jpg
storage_upload_slave_by_filename
group_name=sl, remote_filename=M00/00/00/wKjVeFvz_9aAcMNOAAEJsJdjMjM460_big.jpg
source ip address: 192.168.213.120
file timestamp=2018-11-20 04:36:38
file size=68016
file crc32=2539860531
example file url: http://192.168.213.120/sl/M00/00/00/wKjVeFvz_9aAcMNOAAEJsJdjMjM460_big.jpg
16.可以看到返回信息中显示图片上传到了这个目录中 /fastdfs/storage/data/00/00
五、安装配置nginx
17.安装nginx的fastdfs模块
进入解压后的这个目录中
/root/fastDFS/fastdfs-nginx-module-1.20/src
修改此目录下的config文件
ngx_addon_name=ngx_http_fastdfs_module
if test -n "${ngx_module_link}"; then
ngx_module_type=HTTP
ngx_module_name=$ngx_addon_name
ngx_module_incs="/usr/local/include"
ngx_module_libs="-lfastcommon -lfdfsclient"
ngx_module_srcs="$ngx_addon_dir/ngx_http_fastdfs_module.c"
ngx_module_deps=
CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
. auto/module
else
HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
CORE_INCS="$CORE_INCS /usr/local/include"
CORE_LIBS="$CORE_LIBS -lfastcommon -lfdfsclient"
CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
fi
把上面这两个地方修改成下面的样子
ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
18.安装nginx支持库
yum -y install gcc-c++
yum -y install pcre pcre-devel
yum -y install zlib zlib-devel
yum -y install openssl openssl-devel
19.在nginx的目录下执行以下的配置(这个配置里包括添加上面的模块)
./configure \
--prefix=/usr/local/nginx \
--pid-path=/var/run/nginx/nginx/pid \
--lock-path=/var/lock/nginx.lock \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
--http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module=/root/fastDFS/fastdfs-nginx-module-1.20/src
添加返回结果
Configuration summary
+ using system PCRE library
+ OpenSSL library is not used
+ using system zlib library
nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/nginx/sbin/nginx"
nginx modules path: "/usr/local/nginx/modules"
nginx configuration prefix: "/usr/local/nginx/conf"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/var/run/nginx/nginx/pid"
nginx error log file: "/var/log/nginx/error.log"
nginx http access log file: "/var/log/nginx/access.log"
nginx http client request body temporary files: "/var/temp/nginx/client"
nginx http proxy temporary files: "/var/temp/nginx/proxy"
nginx http fastcgi temporary files: "/var/temp/nginx/fastcgi"
nginx http uwsgi temporary files: "/var/temp/nginx/uwsgi"
nginx http scgi temporary files: "/var/temp/nginx/scgi"
19.进入nginx目录 使用命令编译和安装(不用/make, 直接make)
make && make install
安装后会在/usr/local目录下有一个nginx文件夹，这个文件夹就是安装后的nginx程序
20.进入fastdfs-nginx-module目录中,复制模块配置(这个是fastdfs和nginx配合的配置文件)
cp mod_fastdfs.conf /etc/fdfs/
21 进入/etc/fdfs/目录中，修改mod_fastdfs.conf文件
basepath=/fastdfs/temp
group_name=sl # (同上面storage的组名一样)
url_have_group_name = true
store_path0=/fastdfs/storage # (同storage的配置目录，这个是用于存储文件的目录)
21.进入nginx的安装目录/usr/local/中,修改它的配置文件nginx.conf
# 添加一个server
server {
listen 88;
server_name localhost;
# 这个/slM00是上面设置的组名和自动生成的MOO前缀
location /sl/M00 {
# 这里是我们添加的模块名
ngx_fastdfs_module;
}
}
22.进入/usr/local/nginx/sbin目录执行相关命令
检查参数 ./nginx -t
[root@localhost nginx]# ./sbin/nginx -t
ngx_http_fastdfs_set pid=23514
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: [emerg] mkdir() "/var/temp/nginx/client" failed (2: No such file or directory)
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed
报错说找不到一个目录(/var/temp/nginx/client)，我们去创建这个目录
mkdir /var/temp/nginx/client -p
在次使用 ./sbin/nginx -t 测试即可通过测试
开启nginx ./nginx
23.关闭防火墙
[root@localhost nginx]# /etc/init.d/iptables stop
iptables: Flushing firewall rules: [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Unloading modules: [ OK ]
24.访问192.168.213.120可以看到nginx默认首页
25.重启所有服务(tracker storage nginx)
访问图片
http://192.168.213.120:88/sl/M00/00/00/wKjVeFvz_9aAcMNOAAEJsJdjMjM460.jpg
成功
26.重启机器后重启服务的四条命令
[root@localhost ~]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@localhost ~]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf
[root@localhost ~]# /usr/local/nginx/sbin/nginx
[root@localhost ~]# /etc/init.d/iptables stop
六、整合springboot
1.在springboot工程中添加fastdfs-client-java的maven依赖
<dependency>
<groupId>com.github.tobato</groupId>
<artifactId>fastdfs-client</artifactId>
<version>1.26.2</version>
</dependency>
2.在工程中添加配置类
/**
* 导入FastDFS-Client组件
*
* @author tobato
*
*/
@Configuration
@Import(FdfsClientConfig.class)
// 解决jmx重复注册bean的问题
@EnableMBeanExport(registration = RegistrationPolicy.IGNORE_EXISTING)
public class ComponetImport {
// 导入依赖组件
}
注意：@EnableMBeanExport解决问题JMX重复注册问题,issue #8 issue #18，不要再配置 spring.jmx.enabled=false，以免影响SpringBoot默认的JMX监控。
3.在application.yml当中配置Fdfs相关参数
# ===================================================================
# 分布式文件系统FDFS配置
# ===================================================================
fdfs:
so-timeout: 1501
connect-timeout: 601
thumb-image: #缩略图生成参数
width: 80
height: 80
tracker-list: #TrackerList参数,支持多个
- 192.168.213.120:22122
4.整合好后就可以在java代码中注入相关的工具进行上传下载操作了
@Autowired
private FastFileStorageClient storageClient;
这个类的方法
storageClient.uploadFile
storageClient.uploadFile2
storageClient.uploadQRCode
storageClient.uploadFace
storageClient.uploadBase64
storageClient.uploadFile
storageClient.deleteFile
storageClient.storageClient
