Linux下文件系统FastDFS集群部署
2018-06-27 本文已影响0人
ProMonkey_LAIN
简介
- FastDFS是一个开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。FastDFS服务端有两个角色：跟踪器（tracker）和存储节点（storage）。跟踪器主要做调度工作，在访问上起负载均衡的作用。
-
存储节点存储文件，完成文件管理的所有功能：就是这样的存储、同步和提供存取接口，FastDFS同时对文件的metadata进行管理。所谓文件的meta data就是文件的相关属性，以键值对（key valuepair）方式表示，如：width=1024，其中的key为width，value为1024。文件metadata是文件属性列表，可以包含多个键值对。
拓扑图
部署
- 环境安装
yum install -y zlib zlib-devel pcre pcre-devel gcc gcc-c++ openssl openssl-devel libevent libevent-devel perl unzip 或者 yum groupinstall 'Development tools' -y
- 下载软件包
下载Libfastcommon
wget https://github.com/happyfish100/libfastcommon/archive/master.zip
下载FastDFS
wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz
下载FastDFS-Nginx模块
wget http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz
下载Nginx
wget http://nginx.org/download/nginx-1.8.0.tar.gz
- 创建数据储存目录
#在tracker_server上创建tracker目录；在storage_server上创建storage目录
mkdir -p /data/fastdfs/{storage,tracker}
- 安装Libfastcommon
Tracker-server和storage-server都需要安装
unzip libfastcommon-master.zip
cd libfastcommon-master
./make.sh
./make.sh install
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
- 安装FastDFS
解压、编译安装FastDFS
tar -xf FastDFS_v5.08.tar.gz
cd FastDFS
sh make.sh && sh make.sh install
cp -pa conf/*.conf /etc/fdfs/
cp -pa conf/mime.types /etc/fdfs/
cd /etc/fdfs/
rm -rf *.sample
chown -R nobody.nobody /data/fastdfs
chown -R nobody.nobody /etc/fdfs
注：这里赋予nobody账号权限是因为nginx用nobody用户启动
- 修改tracker.conf配置
disabled=false
bind_addr=10.12.8.42
port=22122
connect_timeout=30
network_timeout=60
base_path=/data/fastdfs/tracker
max_connections=256
accept_threads=1
work_threads=4
store_lookup=2
store_group=group1
store_server=0
store_path=0
download_server=0
reserved_storage_space = 10%
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
trunk_create_file_advance = false
trunk_create_file_time_base = 02:00
trunk_create_file_interval = 86400
trunk_create_file_space_threshold = 20G
trunk_init_check_occupying = false
trunk_init_reload_from_binlog = false
trunk_compress_binlog_min_interval = 0
use_storage_id = false
storage_ids_filename = storage_ids.conf
id_type_in_filename = ip
store_slave_file_use_link = false
rotate_error_log = false
error_log_rotate_time=00:00
rotate_error_log_size = 0
log_file_keep_days = 0
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.server_port=80
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
- 修改storage.conf配置
disabled=false
group_name=group1(这里storage属于group1,如有多个group根据实际情况填写)
bind_addr=
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/data/fastdfs/storage
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/data/fastdfs/storage
subdir_count_per_path=256
tracker_server=10.12.8.42:22122(tracker地址,可写多个)
tracker_server=10.12.8.44:22122(tracker地址,可写多个)
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=80
- 修改mod_fastdfs.conf配置
connect_timeout=2
network_timeout=30
base_path=/data/fastdfs/storage
load_fdfs_parameters_from_tracker=true
storage_sync_file_max_delay = 86400
use_storage_id = false
storage_ids_filename = storage_ids.conf
tracker_server=10.12.8.42:22122
storage_server_port=23000
group_name=group1
url_have_group_name = true
store_path_count=1
store_path0=/data/fastdfs/storage
log_level=info
log_filename=
response_mode=proxy
if_alias_prefix=
flv_support = true
flv_extension = flv
group_count = 0
- 安装Nginx和Fastdfs-Nginx模块
tar -zxf fastdfs-nginx-module_v1.16.tar.gz && tar -zxf nginx-1.10.1.tar.gz
cd fastdfs-nginx-module/src/
cp -r mod_fastdfs.conf /etc/fdfs/
vim /soft/FastDFS/fastdfs-nginx-module/src/config
#修改config文件如下
更改前：“CORE_INCS="$CORE_INCS /usr/local/include/fastdfs /usr/local/include/fastcommon/"
CORE_LIBS="$CORE_LIBS -L/usr/local/lib -lfastcommon -lfdfsclient"”
更改后：“CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/local/include/fastcommon/"
CORE_LIBS="$CORE_LIBS -L/usr/lib64 -lfastcommon -lfdfsclient"”
或者
ln -sv /usr/incloud/fastcommon /usr/local/incloud/fastcommon
ln -sv /usr/incloud/fastdfs /usr/local/incloud/fastdfs
ln -sv /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
#安装nginx
cd nginx-1.10.1
./configure --add-module=/soft/FastDFS/fastdfs-nginx-module/src/ --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module
make && make install
-
vhost/*.conf配置
server {
listen 80 default;
server_name 10.12.8.42;
access_log logs/access.v5ehomepay.log main;
error_log logs/error.v5ehomepay.log;
location /nginx-status {
stub_status on;
access_log on;
}
location /group1/M00 {
root /data/fastdfs/storage/data/;
ngx_fastdfs_module;
}
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js)$ {
# fastcgi_cache cache_one; #nginx.conf 开启cache才行，要不然启动报错
fastcgi_cache_valid 200 10m;
fastcgi_cache_valid 304 3m;
fastcgi_cache_valid 301 302 1h;
fastcgi_cache_valid any 1m;
fastcgi_cache_min_uses 1;
fastcgi_cache_use_stale error timeout invalid_header http_500;
fastcgi_cache_key $host$request_uri;
access_log off;
}
error_page 403 404 /40x.html;
location = /40x.html root html;
}
error_page 404 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
- 启动进程，并查看端口
/etc/init.d/fdfs_trackerd start
/etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
#查看进程
netstat -npl|grep -E "nginx|fdfs"
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 27454/fdfs_storaged
tcp 0 0 10.12.8.42:22122 0.0.0.0:* LISTEN 14179/fdfs_trackerd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 14201/nginx
ps:注意事情：Centos 7.1版本中trackerd和storaged启动失败时，做以下操作
ln -s /usr/bin/fdfs_trackerd /usr/local/bin
ln -s /usr/bin/fdfs_storaged /usr/local/bin
ln -s /usr/bin/stop.sh /usr/local/bin
ln -s /usr/bin/restart.sh /usr/local/bin
- 测试
echo 'test fastDFS!!!' > test.htm
vim /etc/fdfs/client.conf
添加tracker_server地址
fdfs_test /etc/fdfs/client.conf upload test.html
This is FastDFS client test program v5.08
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2016-08-03 15:40:38] DEBUG - base_path=/data/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
tracker_query_storage_store_list_without_group:
server 1. group_name=, ip_addr=10.12.8.42, port=23000
server 2. group_name=, ip_addr=10.12.8.43, port=23000
group_name=group1, ip_addr=10.12.8.43, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51.html
source ip address: 10.12.8.43
file timestamp=2016-08-03 15:40:38
file size=16
file crc32=2988654052
example file url: http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51.html
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
source ip address: 10.12.8.43
file timestamp=2016-08-03 15:40:38
file size=16
file crc32=2988654052
example file url: http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
curl http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
test fastDFS!!!
