Nginx+keepalived+Tomcat集群架构

标签：local   主机   

keepalived实现nginx发生故障时，自动切换，实现nginx反向代理的高可用。

环境：

CentOS6.4  64bit；

Jdk6.tar.gz

Aapache-tomcat-6.0.41.tar.gz

nginx-1.4.2.tar.gz 

主机规划：

keepalived-master:192.168.115.10

keepalived-slave: 192.168.115.20

tomcat1         : 192.168.115.30

tomcat2         : 192.168.115.40

JDK安装：

tar  zxvf   JDK6.tar.gz

vi /etc/profile.d/jdk.sh

export JAVA_HOME=/usr/local/jdk6

export CALASS_PATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export JAVA_BIN=$JAVA_HOME/bin

export PATH=$PATH:$JAVA_HOME/bin

Tomcat安装：

tar zxvf  Aapache-tomcat-6.0.41.tar.gz  

mv  ./tomcat   /usr/local/tomcat1

cp  -r  /usr/local/tomcat1   /usr/local/tomcat2

修改tomcat2中server.xml定义的三个端口：

shutdown端口"8005

ajp端口：    8009

http端口：   8080

nginx的安装：

tar zxvf  pcre-8.12.zip

./configure

make && make install

yum groupinstall "Development Tools"

yum install openssl  openssl-devel

tar zxvf  nginx-1.4.2.tar.gz 

cd nginx-1.4.2

./configure  --prefix=/usr --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-http_ssl_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre 

make 

make install

#########Nginx启动脚本####

#!/bin/sh

#

# nginx - this script starts and stops the nginx daemon

#

# chkconfig:   - 85 15 

# description:  Nginx is an HTTP(S) server, HTTP(S) reverse \

#               proxy and IMAP/POP3 proxy server

# processname: nginx

# config:      /etc/nginx/nginx.conf

# config:      /etc/sysconfig/nginx

# pidfile:     /var/run/nginx.pid

# Source function library.

. /etc/rc.d/init.d/functions

# Source networking configuration.

. /etc/sysconfig/network

# Check that networking is up.

[ "$NETWORKING" = "no" ] && exit 0

nginx="/usr/sbin/nginx"

prog=$(basename $nginx)

NGINX_CONF_FILE="/etc/nginx/nginx.conf"

[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx

lockfile=/var/lock/subsys/nginx

make_dirs() {

   # make required directories

   user=`nginx -V 2>&1 | grep "configure arguments:" | sed ‘s/[^*]*--user=\([^ ]*\).*/\1/g‘ -`

   options=`$nginx -V 2>&1 | grep ‘configure arguments:‘`

   for opt in $options; do

       if [ `echo $opt | grep ‘.*-temp-path‘` ]; then

           value=`echo $opt | cut -d "=" -f 2`

           if [ ! -d "$value" ]; then

               # echo "creating" $value

               mkdir -p $value && chown -R $user $value

           fi

       fi

   done

}

start() {

    [ -x $nginx ] || exit 5

    [ -f $NGINX_CONF_FILE ] || exit 6

    make_dirs

    echo -n $"Starting $prog: "

    daemon $nginx -c $NGINX_CONF_FILE

    retval=$?

    echo

    [ $retval -eq 0 ] && touch $lockfile

    return $retval

}

stop() {

    echo -n $"Stopping $prog: "

    killproc $prog -QUIT

    retval=$?

    echo

    [ $retval -eq 0 ] && rm -f $lockfile

    return $retval

}

restart() {

    configtest || return $?

    stop

    sleep 1

    start

}

reload() {

    configtest || return $?

    echo -n $"Reloading $prog: "

    killproc $nginx -HUP

    RETVAL=$?

    echo

}

force_reload() {

    restart

}

configtest() {

  $nginx -t -c $NGINX_CONF_FILE

}

rh_status() {

    status $prog

}

rh_status_q() {

    rh_status >/dev/null 2>&1

}

case "$1" in

    start)

        rh_status_q && exit 0

        $1

        ;;

    stop)

        rh_status_q || exit 0

        $1

        ;;

    restart|configtest)

        $1

        ;;

    reload)

        rh_status_q || exit 7

        $1

        ;;

    force-reload)

        force_reload

        ;;

    status)

        rh_status

        ;;

    condrestart|try-restart)

        rh_status_q || exit 0

            ;;

    *)

        echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"

        exit 2

esac

#########Nginx启动脚本####

安装keepalived

 yum install popt-devel

 tar  zxvf keepalived-1.1.19.tar.gz

 ./configure --prefix=/usr/local/keepalived

 make && make install

规划配置：

mkdir  /etc/keepalived

cp  /usr/local/keepalived/sbin/keepalived   /usr/sbin/

cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d

cp /usr/local/keepalived/etc/keepalived/keepalived.conf  /etc/keepalived/

修改配置文件：

! Configuration File for keepalived

global_defs {

   notification_email {

     1075841124@qq.com

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

vrrp_instance VI_1 {

    state MASTER        主为MASTER。备为BACKUP

    interface eth0

    virtual_router_id 51  主和备相同

    priority 100           主高备低

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.115.222

        192.168.115.111

        192.168.115.99

    }

}

观察:

MASTER:

/etc/init.d/keepalived start

ip addr show看到如下信息：

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:7e:20:10 brd ff:ff:ff:ff:ff:ff

    inet 192.168.115.10/24 brd 192.168.115.255 scope global eth0

    inet 192.168.115.222/32 scope global eth0

    inet 192.168.115.111/32 scope global eth0

    inet 192.168.115.99/32 scope global eth0

    inet6 fe80::20c:29ff:fe7e:2010/64 scope link 

       valid_lft forever preferred_lft forever

BACKUP:

/etc/init.d/keepalived start

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff

    inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0

    inet6 fe80::20c:29ff:fe05:ffb3/64 scope link 

       valid_lft forever preferred_lft forever

停止master上的keepalived后在观察backup：

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff

    inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0

    inet 192.168.115.222/32 scope global eth0

    inet 192.168.115.111/32 scope global eth0

    inet 192.168.115.99/32 scope global eth0

    inet6 fe80::20c:29ff:fe05:ffb3/64 scope link 

       valid_lft forever preferred_lft forever

配置nginx代理tomcat

http模块中添加：

upstream  tomcat {

        server 192.168.115.30:8080 weight=3;

        server 192.168.115.30:8081 weight=3;

        server 192.168.115.40:8080 weight=3;

        server 192.168.115.40:8081 weight=3;

location / {

             proxy_pass http://tomcat;

           }

Nginx配置完毕。此时测试：

192.168.115.10：MASTER

            /etc/init.d/keepalived start

            server nginx start

192.168.115.20:SLAVE

            /etc/init.d/keepalived start

            server nginx start

后端tomcat全部打开。

注意此时，在keepalived上配置的虚拟IP是：

192.168.115.222

192.168.115.111

192.168.115.99

在浏览器中访问任何一个VIP，nginx都能代理到后方。并且能正常访问到TOMCAT主页面。

现在把MASTER上的nginx stop，观察

192.168.115.10 MASTER：

     service  nginx stop

ip addr show 观察：发现VIP消失

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:7e:20:10 brd ff:ff:ff:ff:ff:ff

    inet 192.168.115.10/24 brd 192.168.115.255 scope global eth0

    inet6 fe80::20c:29ff:fe7e:2010/64 scope link 

       valid_lft forever preferred_lft forever

于是观察BACKUP上，ip addr show 观察已经获取VIP了。

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff

    inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0

    inet 192.168.115.222/32 scope global eth0

    inet 192.168.115.111/32 scope global eth0

    inet 192.168.115.99/32 scope global eth0

    inet6 fe80::20c:29ff:fe05:ffb3/64 scope link 

       valid_lft forever preferred_lft forever

再用浏览器访问VIP，看是否能问到tomcat页面。

依然能访问到，说明keepalived起到作用了。

这就是keepalived双机热备，故障转移的表现功能所在。在Nginx+Keepalived的架构上，nginx双机实现了反向代理的高可用。一台nginx挂掉后，依然不会影响访问。前端负载均衡已经解决了，但是后端的tomcat服务器挂掉后，该如何呢？keepalived是无法做到对后端服务器的健康检测的。这需要Nginx的功能。

本文出自 “常用文档” 博客，请务必保留此出处http://yujianglei.blog.51cto.com/7215578/1566730

Nginx+keepalived+Tomcat集群架构

标签：local   主机   

原文地址：http://yujianglei.blog.51cto.com/7215578/1566730