1Nginx+fastdfs分布式文件存储

标签：

准备，将所需的软件传到服务器上，服务器的列表如下：

1. 首先切换到root用户，命令是：

1. 删除系统自带的低版本的libevent , 注意：这里的tracker和storage端都做相同的操作。

3  安装libevent2.0.21,可以通过.wget的方式进行下来所需要的tar包

4 安装Fastdfs

===================以上是Tracker和Storage都需要安装的部分======================

5 配置Tracker（它是单独一台电脑上的，并且在使用storage服务器的时候，此端要先启动）

注意Tracker服务器IP地址不能是127.0.0.1，Tacker不能和Storage在一台电脑上。

注意的时候，最后运行Tracker的时候，要关闭防火墙，也就是说要执行service iptables stop

===============================================================================

6. 配置storage服务器，注意的是storage和Nginx是在另外一台电脑上的。

1> 创建Storage目录保存运行日志及其data数据

         mkdir-m 777 -p /home/fastdfs/storage

2> 修改storage.conf配置

         vim/usr/local/fastdfs/conf/storage.conf

         #修改内容如下所示

         #the name of the group this storage server belongs to

         group_name=group1#设置组名

         #the name of the group this storage server belongs to

         #the storage server port #the storage server port

         port=23000

         #the base path to store data and log files #日志目录

         base_path=/home/yuqing/fastdfs-> /home/fastdfs/storage

         #store_path#, based 0, if store_path0 not exists, it‘s value is base_path #data数据存储目录

         #the paths must be exist

         store_path0=/home/fastdfs/storage

         #tracker_server can ocur more than once, and tracker_server format is

         #  "host:port", host can be hostnameor ip address

         tracker_server=192.168.209.121:22122->192.168.1.11:22122  这里192.168.1.11:22122表示的是tracker所在电脑的ip地址

 3> 编辑启动脚本

         vim/etc/init.d/fdfs_storaged

         #启动脚本内容如下

         #!/bin/bash

         #

         #fdfs_storaged Starts fdfs_storaged

         #

         #

         #chkconfig: 2345 99 01

         #description: FastDFS storage server

         ###BEGIN INIT INFO

         #Provides: $fdfs_storaged

         ###END INIT INFO

         #Source function library.

         ./etc/init.d/functions

         FastDfs=‘/usr/local/fastdfs‘

         CONF="$FastDfs/conf/storage.conf"

         if[ ! -f $CONF ]; then

           echo "file $CONF does not exist!"

           exit 2

         fi

         PRG="$FastDfs/bin/fdfs_storaged"

         if[ ! -f $PRG ]; then

           echo "file $PRG does not exist!"

           exit 2

         fi

         Stop="$FastDfs/bin/stop.sh"

         if[ ! -f $Stop ]; then

           echo "file $Stop does not exist!"

           exit 2

         fi

         Restart="$FastDfs/bin/restart.sh"

         if[ ! -f $Restart ]; then

           echo "file $Restart does notexist!"

           exit 2

         fi

         RETVAL=0

         start(){

             echo -n $"Starting FastDFS storageserver: "

             $PRG $CONF &

             RETVAL=$?

             echo

             return $RETVAL

         }

         stop(){

             echo -n $"Stop FastDFS storage server:"

             $Stop $PRG $CONF

             RETVAL=$?

             return $RETVAL

         }

         rhstatus(){

             status fdfs_storaged

         }

         restart(){

                   $Restart$PRG $CONF &

         }

         case"$1" in

           start)

             start

             ;;

           stop)

             stop

             ;;

           status)

             rhstatus

             ;;

           restart|reload)

             restart

             ;;

           condrestart)

             restart

             ;;

           *)

             echo $"Usage: $0{start|stop|status|restart|condrestart}"

             exit 1

         esac

         exit$?

4> 给启动脚本增加权限

         chmod777 /etc/init.d/fdfs_storaged

5> 启动storage  （注意的是，启动storage的时候要先启动storage）

         servicefdfs_storaged restart

         #正常情况，接下来会出现很多mkdirdata path，这是系统在创建数据目录

6> 启动成功，加入开机启动

         vim/etc/rc.d/rc.local

         #加入内容如下

         servicefdfs_storaged start

7. 安装nginx（仅正在有Storage的电脑上安装）

1> 创建nginx日志目录

         mkdir-m 777 -p /home/www/logs

2> 安装nginx必需的库：zlib-developenssl-devel pcre

     yum -y install zlib-devel openssl-devel

3> 手动安装pcre

     1> wgetftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.36.tar.gz

     2> tar -zxvf pcre-8.36.tar.gz

     3> cd pcre-8.36

     4> ./configure

     5> make && make install

     6> ln -s /usr/local/lib/libpcre.so.1 /lib

4> 安装nginx

     1> wget http://nginx.org/download/nginx-1.5.6.tar.gz

     2> tar -zxvf nginx-1.5.6.tar.gz

     3> cd nginx-1.5.6

     4> ./configure --prefix=/usr/local/nginx --with-http_stub_status_module--with-http_ssl_module   （注意：这里的--with-http_ssl_module可以不加）

     5> make && make install

5> 检查nginx配置是否正确

      /usr/local/nginx/sbin/nginx -t

6> 查看nginx编译选项

      /usr/local/nginx/sbin/nginx -V

7> 编辑启动脚本

      vim /etc/init.d/nginxd

      #启动脚本内容如下

         #!/bin/sh

         #

         #nginx - this script starts and stops the nginx daemin

         #

         #description:  Nginx is an HTTP(S) server,HTTP(S) reverse \

         #               proxy and IMAP/POP3 proxy server

         #processname: nginx

         #config:     /usr/local/nginx/conf/nginx.conf

         #pidfile:     /usr/local/nginx/nginx.pid

         #Source function library.

         ./etc/rc.d/init.d/functions

         #Source networking configuration.

         ./etc/sysconfig/network

         #Check that networking is up.

         ["$NETWORKING" = "no" ] && exit 0

         #必填

         nginx="/usr/local/nginx/sbin/nginx"

         prog=$(basename$nginx)

         #必填

         NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"

         [-f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx

         lockfile=/var/lock/subsys/nginx

         start(){

             [ -x $nginx ] || exit 5

             [ -f $NGINX_CONF_FILE ] || exit 6

             echo -n $"Starting $prog: "

             daemon $nginx -c $NGINX_CONF_FILE

             retval=$?

             echo

             #service php-fpm start

             [ $retval -eq 0 ] && touch$lockfile

             return $retval

         }

         stop(){

             echo -n $"Stopping $prog: "

             $nginx -s stop

             echo_success

             retval=$?

             echo

             #service php-fpm stop

             [ $retval -eq 0 ] && rm -f$lockfile

             return $retval

         }

         restart(){

             stop

             start

         }

         reload(){

             configtest || return $?

             echo -n $"Reloading $prog: "

             $nginx -s reload

             RETVAL=$?

             echo

         }

         force_reload(){

             restart

         }

         configtest(){

           $nginx -t -c $NGINX_CONF_FILE

         }

         version(){

           $nginx -V

         }

         rh_status(){

             status $prog

         }

         rh_status_q(){

             rh_status >/dev/null 2>&1

         }

         case"$1" in

             start)

                   rh_status_q&& exit 0

                   $1

                   ;;

             stop)

                   rh_status_q|| exit 0

                   $1

                   ;;

             restart|configtest|version)

                   $1

                   ;;

             reload)

                   rh_status_q|| exit 7

                   $1

                   ;;

             force-reload)

                   force_reload

                   ;;

             status)

                   rh_status

                   ;;

             condrestart|try-restart)

                   rh_status_q|| exit 0

                       ;;

             *)

                   echo$"Usage: $0{start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest|version}"

                   exit2

         esac

8> 给启动脚本增加权限

         chmod777 /etc/init.d/nginxd

9> 启动nginx

         servicenginxd restart

         如果启动失败提示找不到库，ldconfig更新库路径

10> 启动成功，加入开机启动

         vim/etc/rc.d/rc.local

         #加入内容如下

         servicenginxd start

8. 安装nginx-module模块（仅在有Storage的电脑上）

1> 下载fastdfs-nginx-module，google被禁了，需要翻墙

         wgethttp://fastdfs.googlecode.com/files/fastdfs-nginx-module_v1.15.tar.gz

2> tar -zxvffastdfs-nginx-module_v1.15.tar.gz;

3> 修改插件配置文件,主要是跟安装fastdfs目录相关

         vim<fastdfs-nginx-module源码目录>/src/config

         #修改内容如下

         ngx_addon_name=ngx_http_fastdfs_module

         HTTP_MODULES="$HTTP_MODULESngx_http_fastdfs_module"

         NGX_ADDON_SRCS="$NGX_ADDON_SRCS$ngx_addon_dir/ngx_http_fastdfs_module.c"

         CORE_INCS="$CORE_INCS/usr/local/fastdfs/include/fastdfs /usr/local/fastdfs/include/fastcommon/"

         CORE_LIBS="$CORE_LIBS-L/usr/local/fastdfs/lib -lfastcommon -lfdfsclient"

         CFLAGS="$CFLAGS-D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE=‘256*1024‘-DFDFS_MOD_CONF_FILENAME=‘\"/usr/local/fastdfs/conf/mod_fastdfs.conf\"‘"

4> 复制mod_fastdfs.conf到/usr/local/fastdfs/conf/目录下

         cp<fastdfs-nginx-module源码目录>/src/mod_fastdfs.conf /usr/local/fastdfs/conf/

5> 将/usr/local/fastdfs/lib加入系统文件/etc/ld.so.conf中（编译时使用的动态链接库)

         vim/etc/ld.so.conf

         #修改内容如下

         /usr/local/fastdfs/lib

6> 更新库文件缓存ld.so.cache

         ldconfig

7> 编译fastdfs-nginx-module模块

     1> cd nginx-1.5.6/

     2> ./configure 此处加上nginx之前的编译参数（使用 /usr/local/nginx/sbin/nginx -V 命令查看）--add-module=<fastdfs-nginx-module源码目录>/src

     3> make && make install

8> 修改mod_fastdfs.conf配置

         vim/usr/local/fastdfs/conf/mod_fastdfs.conf

         #修改内容如下

         #if load FastDFS parameters from tracker server

         #since V1.12

         #default value is false

         load_fdfs_parameters_from_tracker=true

         #FastDFS tracker_server can ocur more than once, and tracker_server format is

         #  "host:port", host can be hostnameor ip address

         #valid only when load_fdfs_parameters_from_tracker is true

         tracker_server=192.168.25.11:22122

         #the port of the local storage server

         #the default value is 23000

         storage_server_port=23000

         #the group name of the local storage server

         group_name=group1#当前storage机器组名

         #if the url / uri including the group name

         #set to false when uri like /M00/00/00/xxx

         #set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx

         #default value is false

         url_have_group_name= true

         #path(disk or mount point) count, default value is 1

         #must same as storage.conf

         store_path_count=1

         #store_path#, based 0, if store_path0 not exists, it‘s value is base_path

         #the paths must be exist

         #must same as storage.conf

         store_path0=/home/fastdfs/storage

         #set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log

         #empty for output to stderr (apache and nginx error_log file)

         log_filename=/home/www/logs/mod_fastdfs.log

 9. 修改nginx.conf配置

     vim /usr/local/nginx/conf/nginx.conf

     #修改内容如下（下面的可以替换原来文件中的所有内容）

         worker_processes  2;

         error_log  /home/www/logs/error.log  notice;

         pid        /home/www/logs/nginx.pid;

         worker_rlimit_nofile5120;

         events{

             use epoll;

             worker_connections  5120;

         }

         http{

             include       mime.types;

             default_type  application/octet-stream;

             sendfile        on;

             tcp_nopush     on;

             keepalive_timeout  60;

             tcp_nodelay on;

             server {

                   listen       80;

                   server_name  localhost;

                   location/group1/M00 {

                       alias /home/fastdfs/storage/data;

                       ngx_fastdfs_module;

                   }

             }

         }

10. 测试（注意：如果不成功，还要修改client.config中的内容）

1.创建测试文件

echo "hello nginx fastdfs" >test.html

2.查看fastdfs自带测试命令使用方法

/usr/local/fastdfs/bin/fdfs_test -?

3.上传测试文件，通过tracker转存到storage服务器里

/usr/local/fastdfs/bin/fdfs_test/usr/local/fastdfs/conf/client.conf upload test.html

得到以下信息

tracker_query_storage_store_list_without_group:

         server1. group_name=, ip_addr=172.16.63.13, port=23000

group_name=group1, ip_addr=172.16.63.13,port=23000

storage_upload_by_filename

group_name=group1,remote_filename=M00/00/00/oYYBAFSSxiuABDlLAAAABncc3SA45.html

source ip address: 172.16.63.13

file timestamp=2014-12-18 20:18:51

file size=6

file crc32=1998380320

file url:http://172.16.63.12:8080/group1/M00/00/00/oYYBAFSSxiuABDlLAAAABncc3SA45.html

storage_upload_slave_by_filename

group_name=group1, remote_filename=M00/00/00/oYYBAFSSxiuABDlLAAAABncc3SA45_big.html

source ip address: 172.16.63.13

file timestamp=2014-12-18 20:18:51

file size=6

file crc32=1998380320

file url:http://172.16.63.12:8080/group1/M00/00/00/oYYBAFSSxiuABDlLAAAABncc3SA45_big.html

4.打开浏览器，查看上传文件

         http://172.16.63.13/group1/M00/00/00/oYYBAFSSwuKAO89DAAAABncc3SA04.html

看到"hello nginx fastdfs"则成功

备注：

1> 由于没有设置好防火墙，同组group到storage之间备份数据失败，关闭防火墙：service iptables stop

2> 至少有一个storage到主机安装nginx

3> mod_fastdfs.conf此配置文件安装路径

4> tracker服务器ip不能是127.0.0.1

1Nginx+fastdfs分布式文件存储

标签：

原文地址：http://blog.csdn.net/tototuzuoquan/article/details/42119789