设为首页收藏本站

LinuxTone | 运维专家网论坛 - 最棒的Linux运维与开源架构技术交流社区!

 找回密码
 注册

QQ登录

只需一步,快速开始

#公告#抱歉,网站将关闭,不再开放。由于PC时代已远逝 。在这个后移动互联网时代,我们继续携手前行,保持对技术的热情。共同构建linuxtone知识星球欢迎加入,一起讨论技术、招聘人才、分享资源。请新老linuxtone人 扫码移步到 知识星球:linuxtone

学习是一种信仰!分享是一种快乐!能力= 心态 * 沟通 * 知识 (你的每一天需要正能量!)

 网站的发展需要你贡献一份力量!希望你能每天坚持看贴1小时,并回答网友的问题!祝你在浏览论坛的过程中取得进步!谢谢!linuxtone加油!大家加油! 友情提示: 你今天学习了吗?你今天进步了吗?少一点抱怨!多一点进步!Life is short ! Why not linuxtone ?  

网站的发展、感谢每位坛友的努力!

查看: 11235|回复: 2

FastDFS的安装和配置 [复制链接]

Rank: 6Rank: 6

签到
32
注册时间
2013-6-24
最后登录
2017-12-6
在线时间
12 小时
阅读权限
70
积分
3160
帖子
20
主题
9
精华
0
UID
38724
发表于 2014-4-16 10:52:59 |显示全部楼层

原文地址:http://www.hylinux.cn/?p=169

fastdfs是一个开源的,高性能的的分布式文件系统,他主要的功能包括:文件存储,同步和访问,设计基于高可用和负载均衡,fastfd非常适用于基于文件服务的站点,例如图片分享和视频分享网站

fastfds有两个角色:跟踪服务和存储服务,跟踪服务控制,调度文件以负载均衡的方式访问;存储服务包括:文件存储,文件同步,提供文件访问接口,同时以key value的方式管理文件的元数据

跟踪和存储服务可以由1台或者多台服务器组成,同时可以动态的添加,删除跟踪和存储服务而不会对在线的服务产生影响,在集群中,tracker服务是对等的

存储系统由一个或多个卷组成,卷与卷之间的文件是相互独立的,所有卷的文件容量累加就是整个存储系统中的文件容量。一个卷可以由一台或多台存储服务器组成,一个卷下的存储服务器中的文件都是相同的,卷中的多台存储服务器起到了冗余备份和负载均衡的作用。在卷中增加服务器时,同步已有的文件由系统自动完成,同步完成后,系统自动将新增服务器切换到线上提供服务。当存储空间不足或即将耗尽时,可以动态添加卷。只需要增加一台或多台服务器,并将它们配置为一个新的卷,这样就扩大了存储系统的容量。

下面将介绍下fastdfs在rhel上的部署过程
tracker服务器:192.168.123.110/24
storage服务器:192.168.123.20/24

一:编译安装

[root@db1 ~]# wget http://cloud.github.com/download ... .0.14-stable.tar.gz  
[root@db1 ~]# tar -zxvpf libevent-2.0.14-stable.tar.gz   
[root@db1 ~]# cd libevent-2.0.14-stable  
[root@db1 libevent-2.0.14-stable]# ./configure --prefix=/usr/local/libevent-2.0.14 && make && make install  
[root@db1 ~]# wget http://fastdfs.googlecode.com/files/FastDFS_v3.02.tar.gz  
[root@db1 ~]# tar -zxvf FastDFS_v3.02.tar.gz   
[root@db1 ~]# cd FastDFS  
[root@db1 FastDFS]# grep -A 4 '/usr/local/FastDFS' make.sh   
TARGET_PREFIX=/usr/local/FastDFS
TARGET_CONF_PATH=/etc/fdfs
WITH_HTTPD=1
WITH_LINUX_SERVICE=1
[root@db1 FastDFS]# ./make.sh C_INCLUDE_PATH=/usr/local/libevent-2.0.14/include LIBRARY_PATH=/usr/local/libevent-2.0.14/lib  
[root@db1 FastDFS]# ./make.sh install  
[root@db1 FastDFS]# ls /etc/fdfs/  
client.conf  http.conf  mime.types  storage.conf  tracker.conf
二:tracker配置文件

[root@db1 ~]# grep -v '^#' /etc/fdfs/tracker.conf |grep -v '^$'  
disabled=false
bind_addr=192.168.123.110
port=22122
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
max_connections=256
work_threads=4
store_lookup=2
store_group=group2
store_server=0
store_path=0
download_server=0
reserved_storage_space = 4GB
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
http.disabled=false
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
http.need_find_content_type=true
[root@db1 ~]# grep -v '^#' /etc/fdfs/http.conf  |grep -v '^$'  
http.default_content_type = application/octet-stream
http.mime_types_filename=/etc/fdfs/mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg
三:启动tracker服务,需要注意tracker.conf文件最后一行为#include httpd.conf

[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf   
/usr/local/FastDFS/bin/fdfs_trackerd: error while loading shared libraries: libevent-
2.0.so.5: cannot open shared object file: No such file or directory
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf  
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf  
[root@db1 ~]# ldconfig  
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[2012-07-04 17:52:25] ERROR - file: tracker_func.c, line: 160, "/home/data/fastdfs"
can't be accessed, error info: No such file or directory
[root@db1 ~]# mkdir -p /home/data/fastdfs  
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf   
[root@db1 ~]# echo $?  
22
[root@db1 ~]# cat  /home/data/fastdfs/logs/trackerd.log   
[2012-07-04 17:52:50] ERROR - file: ../common/fdfs_http_shared.c, line: 128, param
"http.mime_types_filename" not exist or is empty
[root@db1 ~]# tail -1 /etc/fdfs/tracker.conf   
#include http.conf
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf   
[root@db1 ~]# echo $?  
0
[root@db1 ~]# ps -ef |grep track  
root      3535     1  0 15:47 ?        00:00:00 /usr/local/FastDFS/bin/fdfs_trackerd
/etc/fdfs/tracker.conf
[root@db1 ~]# netstat -ntpl |grep fdfs
tcp        0      0 192.168.123.110:22122       0.0.0.0:*                   LISTEN
3535/fdfs_trackerd
tcp        0      0 192.168.123.110:8080        0.0.0.0:*                   LISTEN
3535/fdfs_trackerd
四:storage配置文件

[root@db2 ~]# grep -v '^#' /etc/fdfs/storage.conf |grep -v '^$'  
disabled=false
group_name=group1
bind_addr=192.168.123.20
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/home/data/fastdfs
max_connections=256
buff_size = 256KB
work_threads=4
disk_rw_separated = true
disk_rw_direct = false
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/home/data/fastdfs
subdir_count_per_path=256
tracker_server=192.168.123.110:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
key_namespace=FastDFS
keep_alive=0
http.disabled=false
httphttp.domain_name=
http.server_port=8888
http.trunk_size=256KB
http.need_find_content_type=true
[root@db2 ~]# grep -v '^#' /etc/fdfs/client.conf  |grep -v '^$'
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
tracker_server=192.168.123.110:22122
log_level=info
http.tracker_server_port=8080
[root@db2 ~]#  grep -v '^#' /etc/fdfs/http.conf |grep -v '^$'  
http.default_content_type = application/octet-stream
http.mime_types_filename=mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg
五:启动storage,需要注意storage.conf文件最后一行为#include httpd.conf

[root@db2 ~]# mkdir -p /home/data/fastdfs  
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf  
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf  
[root@db2 ~]# ldconfig  

[root@db2 ~]# tail -2 /etc/fdfs/storage.conf   
#use "#include" directive to include HTTP other settings
#include http.conf
[root@db2 ~]# /usr/local/FastDFS/bin/fdfs_storaged /etc/fdfs/storage.conf   
mkdir data path: 00 ...
mkdir data path: 01 ...
mkdir data path: 02 ...
———输出省略————
data path: /home/data/fastdfs/data, mkdir sub dir done.
[root@db2 ~]# ps -ef |grep fdfs  
root     14451     1  0 16:15 ?        00:00:00 /usr/local/FastFDS/bin/fdfs_storaged
/etc/fdfs/storage.conf
root     14468  8238  0 16:16 pts/1    00:00:00 grep fdfs
[root@db2 ~]# netstat -ntpl |grep fdfs  
tcp        0      0 192.168.123.20:8888         0.0.0.0:*                   LISTEN
14451/fdfs_storaged
tcp        0      0 192.168.123.20:23000        0.0.0.0:*                   LISTEN
14451/fdfs_storaged
五:测试文件上传

[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload /etc/passwd
This is FastDFS client test program v3.02
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.[2012-07-30 16:25:59] INFO - base_path=/home/data/fastdfs, connect_timeout=30,
network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group:
server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:25:59
file size=1832
file crc32=295790097
file url: http://192.168.123.110:8080/grou ... tAAAHKBGhZhE8065783
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783_big
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:25:59
file size=1832
file crc32=295790097
file url: http://192.168.123.110:8080/grou ... HKBGhZhE8065783_big

[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload 2.jpg
This is FastDFS client test program v3.02
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.

[2012-07-30 16:29:22] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group:
server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472.jpg
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:29:22
file size=71850
file crc32=3833797383
file url: http://192.168.123.110:8080/grou ... sAAEYquSDGwc472.jpg
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472_big.jpg
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:29:22
file size=71850
file crc32=3833797383
file url: http://192.168.123.110:8080/grou ... YquSDGwc472_big.jpg


该贴已经同步到 qhy02he的微博

Rank: 8Rank: 8

注册时间
2011-9-16
最后登录
2019-3-24
在线时间
848 小时
阅读权限
90
积分
232327
帖子
2158
主题
2
精华
0
UID
15647
发表于 2014-4-17 09:15:46 |显示全部楼层
感谢楼主分享!

使用道具 举报

Rank: 8Rank: 8

注册时间
2010-10-15
最后登录
2016-4-3
在线时间
1023 小时
阅读权限
90
积分
175563
帖子
696
主题
14
精华
0
UID
10623
发表于 2014-4-17 16:12:59 |显示全部楼层
:lol:lol
FastDFS3.0.2
LZ你这是忽悠群众啊~~~

使用道具 举报

您需要登录后才可以回帖 登录 | 注册

IT运维专家网感谢您的支持

合作联系: QQ:67888954/MSN:cnseek@msn.com/mail:netseek@linuxtone.org

Archiver|手机版|感谢所有关心和支持过LinuxTone的朋友们 转载本站内容请注明原作者名及出处 ( 京ICP备08103151 )   |

GMT+8, 2019-3-25 01:11 , Processed in 0.021606 second(s), 12 queries , Apc On.

Powered by Discuz! X2 Licensed

© 2001-2011 Comsenz Inc.

回顶部