【FastDFS分布式文件系统之一】:搭建、部署、配置

        最近要提高小文件(100KB左右)的存储性能(主要是写入性能),想到了使用FastDFS因此需要对其性能做一个简单的测试。


1.安装FastDFS

cd /opt && mkdir fastdfs && cd fastdfs
git clone https://github.com/happyfish100/fastdfs.git
git clone https://github.com/happyfish100/libfastcommon.git
cd /opt/fastdfs/libfastcommon
./make.sh && ./make.sh install
cd /opt/fastdfs/fastdfs
./make.sh && ./make.sh install
安装成功后,FastDFS安装在/usr/bin中。配置文件在/etc/fdfs中


2.修改配置文件

配置文件放在/etc/fdfs下面:

client配置文件:/etc/fdfs/client.conf.sample

storage配置文件:/etc/fdfs/storage.com.sample

tracker配置文件:/etc/fdfs/tracker.com.sample

cd /etc/fdfs
mv client.conf.sample client.conf
mv storage.conf.sample storage.conf
mv tracker.conf.sample tracker.conf
client:node1

tracker server:node2 / node3

group1:storage server1(node4) /storage server2(node5) /storage server3(node6)

group2:storage server4(node7) /storage server5(node6) /storage server6(node8)

修改client.conf:

base_path=/mnt/xfsd/fastdfs/client
tracker_server=node2:22122
tracker_server=node3:22122
#其他都采用默认
其中base_path是客户端的数据和日志写入的目录,这个目录必须指定且提前创建好。

修改storage.conf:

base_path=/mnt/xfsd/fastdfs/storage
max_connections=2048
accept_threads=4
work_threads=8
tracker_server=node2:22122
tracker_server=node3:22122
log_level=info
#其他都采用默认
max_connections:并发最大请求数,默认256,如果设置的比较小,那么如果并发请求达到300以上,就会造成失败,如果设置的太大(20480)又会造成storage server启动失败,不知道为什么。具体的参数配置可以见配置文件的注释。

修改tracker.conf:

base_path=/mnt/xfsd/fastdfs/tracker
max_connections=2048
accept_threads=4
work_threads=8
log_level=info
use_trunk_file = true
值得一提的是use_trunk_file来进行小文件的合并,默认合并成为64MB的块。

3.启动服务

服务的启动,重启,状态查询等。

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf

service fdfs_storaged status
service fdfs_storaged start
service fdfs_storaged stop
service fdfs_storaged restart

service fdfs_trackerd status
service fdfs_trackerd start
service fdfs_trackerd restart
service fdfs_trackerd stop

4.检查服务

/usr/bin/fdfs_monitor /etc/fdfs/storage.conf
输出如下信息:

[2016-03-08 17:40:36] DEBUG - base_path=/mnt/xfsd/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=2, server_index=1

tracker server is XXXXXX:22122

group count: 2

Group 1:
group name = group1
disk total space = 6654860 MB
disk free space = 6430591 MB
trunk free space = 127 MB
storage server count = 3
active server count = 3
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 3481

	Storage 1:
		id = XXX.119
		ip_addr = XXX.119  ACTIVE
		http domain = 
		version = 5.08
		join time = 2016-03-07 21:46:38
		up time = 2016-03-07 21:46:38
		total storage = 6654860 MB
		free storage = 6431077 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888\
	
	Storage 2:
    ……

    Storage 3:
	……

Group 2:
group name = group2
disk total space = 6654860 MB
disk free space = 6429913 MB
trunk free space = 152 MB
storage server count = 3
active server count = 3
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 2
current trunk file id = 3501

	Storage 1:
		id = XXX.122
		ip_addr = XXX.122  ACTIVE
		http domain = 
		version = 5.08
		join time = 2016-03-07 21:50:57
		up time = 2016-03-07 21:50:57
		total storage = 6654860 MB
		free storage = 6429917 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888
		current_write_path = 0
		source storage id = 
		if_trunk_server = 1
	
	Storage 2:
    ……

    Storage 3:
	……

5.数据操作

在client端执行测试:

上传upload操作:

/usr/bin/fdfs_test /etc/fdfs/client.conf upload /data/files/bak/smallfile0
【FastDFS分布式文件系统之一】:搭建、部署、配置_第1张图片
下载download操作:

【FastDFS分布式文件系统之一】:搭建、部署、配置_第2张图片


6.重新部署方法

      由于测试需要,将replica 2变成replica 3,那么只需要将tracer server以及storage server端的base_path下面的client,storage,tracker文件夹都删除掉,然后重新创建文件夹,重启各个server服务就好了,无需卸载,重新编译安装等。


Author:忆之独秀

Email:[email protected]

注明出处:http://blog.csdn.net/lavorange/article/details/50814086 




你可能感兴趣的:(分布式文件系统,fastDFS,分布式存储)