Nginx+keepalived双机热备

标签：nginx   ha   keepalive   

Nginx是一款轻量级的Web 服务器/反向代理服务器及电子邮件（IMAP/POP3）代理服务器，并在一个BSD-like 协议下发行。其特点是占有内存少，并发能力强，事实上nginx的并发能力确实在同类型的网页服务器中表现较好，中国大陆使用nginx网站用户有：百度、京东、新浪、网易、腾讯、淘宝等。

二、测试环境

下面拿2台虚拟机进行环境测试,实验环境为centos6.6 x86_64,具体用途和ip如下

三、安装nginx

这边给大家使用源码包来安装nginx

groupadd nginx

useradd -g nginx -s /sbin/nologin nginx

mkdir /var/log/nginx&&chown -R nginx:nginx /var/log/nginx

mkdir /usr/local/www&&chown -R nginx:nginx /usr/local/www

安装nginx所需pcre库

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz

tar zxvf pcre-8.38.tar.gz

cd pcre-8.38

./configure

make && make install

wget http://nginx.org/download/nginx-1.14.0.tar.gz

cd nginx-1.14.0

./configure --prefix=/usr/local/nginx--user=nginx --group=nginx \

--with-http_ssl_module \

--with-http_stub_status_module \

 --with-http_flv_module  \

--with-http_gzip_static_module

make && make install

ln -s /usr/local/lib/libpcre.so.1 /lib64

使用/usr/local/nginx/sbin/nginx来启动nginx服务

访问下默认nginx页面

四、修改linux文件句柄数

使用ulimit -a查看默认为1024

open files                      (-n) 1024

1、直接在终端修改 ulimit -SHn 65535

2、修改linux系统参数

vi /etc/security/limits.conf 添加保存后，重启机器就永久生效了

* soft nofile 65535

* hard nofile 65535

五、配置nginx
这边给大家提供了一个最简单的配置文件给大家参考

cat /usr/local/nginx/conf/nginx.conf
user  nginx nginx;
worker_processes  8;

error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

pid        /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    #charset gb2312;
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    client_max_body_size 8m;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  60;
    tcp_nodelay on;
    gzip  on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types text/plain application/x-javascript text/css application/xml;
    gzip_vary on;

    server {
        listen       80;
        server_name  192.168.214.76;
        index index.html index.htm;
        root /usr/local/www;
        #charset koi8-r;
location /nginx_status {
        stub_status on;
        access_log   off;
                }
        access_log  /var/log/nginx/$server_name.log  combined;
}
}

六、安装keepalived

yum install -y keepalived

chkconfig keepalived on

注：在centos7系列系统中开机自动启动使用systemctl enable keepalived

七、keepalived文件配置

查看192.168.214.76主keepalived配置

cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     charles@test.com
   }
   notification_email_from info@test.com
   smtp_server mail.test.com
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {                  
script "/usr/local/scripts/nginx_pid.sh"     
interval 2                  
weight 2 
 }  

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_interface {
            eth0
  }
track_script {           
chk_http_port          
}      
virtual_ipaddress {         
192.168.214.70     
}  
}

查看192.168.214.77备的keepalived配置

! Configuration File for keepalived

global_defs {
   notification_email {
     charles@test.com
   }
   notification_email_from info@test.com
   smtp_server mail.test.com
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {                  
script "/usr/local/scripts/nginx_pid.sh"                  
interval 2                  
weight 2 
 }

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99 
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_interface {
           eth0
    }
track_script {           
chk_http_port          
}      

virtual_ipaddress {         
192.168.214.70    
}  
}

最后附上Nginx的检测脚本，当主服务器探测到nginx服务停止后，尝试开启nginx服务，失败后关闭主服务器的keepalived服务，把vip切换到备上

cat nginx_pid.sh 
#!/bin/bash  
A=`ps -C nginx --no-header |wc -l`                 
if [ $A -eq 0 ];then                                                         
               /usr/local/nginx/sbin/nginx                  
              sleep 3                  
              if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then                         
                     killall keepalived                  
              fi  
fi

八、启动keepalived服务及查看相关信息

在192.168.214.76上通过ip addr 查看，vip192.168.214.70已经绑定在eth0网口上了

[root@localhost scripts]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN

   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

   inet 127.0.0.1/8 scope host lo

   inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 52:54:00:55:b2:d4 brd ff:ff:ff:ff:ff:ff

   inet 192.168.214.76/24 brd 192.168.214.255 scope global eth0

   inet 192.168.214.70/32 scope global eth0

   inet6 fe80::5054:ff:fe55:b2d4/64 scope link

      valid_lft forever preferred_lft forever

3: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 52:54:00:85:11:95 brd ff:ff:ff:ff:ff:ff

   inet 192.168.211.76/24 brd 192.168.211.255 scope global eth1

   inet6 fe80::5054:ff:fe85:1195/64 scope link

      valid_lft forever preferred_lft forever

在214.76上查看日志信息，看到已成功进入keepalived主机模式

[root@localhost scripts]# tail -f/var/log/messages

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Registering Kernel netlink command channel

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Opening file '/etc/keepalived/keepalived.conf'.

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Configuration is using : 7681 Bytes

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Using LinkWatch kernel netlink reflector...

May 17 16:29:01 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Transition to MASTER STATE

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) EnteringMASTER STATE

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) setting protocol VIPs.

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 192.168.214.70

May 17 16:29:02 localhostKeepalived_healthcheckers[1747]: Netlink reflector reports IP 192.168.214.70 added

May 17 16:29:07 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.214.70

在214.77上查看日志信息，看到已成功进入keepalived备机模式

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Opening file '/etc/keepalived/keepalived.conf'.

May 17 16:29:06 localhost Keepalived_vrrp[22914]:Configuration is using : 66478 Bytes

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringBACKUP STATE

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]

May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...

九、keepalived测试

使用vip192.168.214.70访问nginx

最后，我们模拟下192.168.214.76nginx服务宕机，且无法通过脚本恢复，看下vip地址是否会漂移过去，nginx页面是否能正常访问。

通过ip addr命令看到vip已漂移到了192.168.214.77nginx服务器上，成功实现了热备。

[root@localhost scripts]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN

   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

   inet 127.0.0.1/8 scope host lo

   inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 52:54:00:1b:a2:11 brd ff:ff:ff:ff:ff:ff

   inet 192.168.214.77/24 brd 192.168.214.255 scope global eth0

   inet 192.168.214.70/32 scope global eth0

   inet6 fe80::5054:ff:fe1b:a211/64 scope link

      valid_lft forever preferred_lft forever

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP>mtu 1500 qdisc pfifo_fast state UP qlen 1000

   link/ether 52:54:00:64:47:7d brd ff:ff:ff:ff:ff:ff

   inet 192.168.211.77/24 brd 192.168.211.255 scope global eth1

   inet6 fe80::5054:ff:fe64:477d/64 scope link

      valid_lft forever preferred_lft forever

在214.77上查看日志信息，看到从之前的备机模式已成功进入keepalived主机模式

[root@localhost scripts]# tail -f/var/log/messages

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...

May 17 16:29:06 localhost Keepalived_vrrp[22914]:VRRP_Instance(VI_1) Entering BACKUP STATE

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]

May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...

May 17 16:49:47 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Transitionto MASTER STATE

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringMASTER STATE

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) setting protocol VIPs.

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70

May 17 16:49:48 localhostKeepalived_healthcheckers[22912]: Netlink reflector reports IP 192.168.214.70added

May 17 16:49:53 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70

扫码关注

Nginx+keepalived双机热备

标签：nginx   ha   keepalive   

原文地址：http://blog.51cto.com/5ydycm/2118114