码迷,mamicode.com
首页 > 其他好文 > 详细

Lvs + Ngnix + Haproxy + Keepalived + Tomcat 实现三种HA软负载均衡和Tomcat Session共享

时间:2015-06-10 08:59:32      阅读:729      评论:0      收藏:0      [点我收藏+]

标签:lvs   haproxy   keepalived   tomcat   nginx   

环境准备:

一、11台测试机器
    
    
    由于在内网测试,需要搭建个内网yum源,方便安装软件。yum所在机器为192.168.33.101

二、待实现功能
    下面分别使用haproxy/nginx/lvs实现HA + 负载均衡,软件环境如下:
        v1:33.81 nginx
        v2:33.82 nginx    
        v3:33.83 tomcat
        v4:33.84 tomcat
        v5:33.85 tomcat
        v6:33.86 tomcat
        v7:33.87 haproxy
        v8:33.88 haproxy
        v9:33.89 lvs
        v10:33.90 lvs


  

1. 安装依赖包

yum -y install pcre-devel
 
yum -y install openssl-devel
 
yum -y install gcc
 
yum -y install lrzsz
 
yum -y install openssh-clients

2. 安装nginx

2.1. 上传、解压、重命名nginx

su - root

1cd /usr/local

2rz -y

3tar -xzvf nginx-1.7.7.tar.gz

4mv nginx-1.7.

1进入解压后的目录,指定安装路径,
 
注:不指定prefix,则可执行文件默认放在/usr /local/bin,库文件默认放在/usr/local/lib,配置文件默认放在/usr/local/etc
 
cd /usr/local/nginx
 
./configure --prefix=/usr/local/nginx --conf-path=/usr/local/nginx/nginx.conf
 
2编译:
 
make
 
3安装:
 
make install
 
4启动
 
/usr/local/nginx/sbin/nginx
 
5查看
 
http://192.168.xx.xxx
 
出现:welcome Nginx,就安装ok了。
 
6停止
 
/usr/local/nginx/sbin/nginx s stop

2.2. 安装nginx

2.3. 安装jdk

1su - root 用户
 
2、进入usr目录
 
cd /usr
 
3、在usr目录下建立java安装目录
 
mkdir m 755 java
 
4、将jdk-6u24-linux-i586.bin拷贝到java目录下
 
rz -y
 
4、安装jdk
 
cd /usr/java
 
chmod 755 jdk-6u24-linux-i586.bin
 
./jdk-6u24-linux-i586.bin
 
注意:如果出现/lib/ld-linux.so.2: bad ELF interpreter:No such file or directory,安装下glic即可: yum install glibc.i686
 
5、安装完毕为他建立一个链接以节省目录长度
 
ln -s /usr/java/jdk1.6.0_24//usr/jdk
 
6、配置环境变量
 
vim /etc/profile
 
添加内容:
 
vi /etc/profile
 
export JAVA_HOME=/usr/jdk
 
export PATH=$PATH:$JAVA_HOME/bin
 
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
 
export JAVA_HOME PATH CLASSPATH
 
7、执行下命令(source命令也称为“点命令”,也就是一个点符号(.)。source命令通常用于重新执行刚修改的初始化文件,使之立即生效,而不必注销并重新登录。)
 
source /etc/profile

2.4. 安装tomcat

1上传、解压:
 
rz -y
 
tar -zxvf apache-tomcat-6.0.37.tar.gz
 
2、重命名:
 
mv apache-tomcat-6.0.37 tomcat

 

2.5. 重新配置nginx

1cd /usr/local/nginx
 
2vi /usr/local/nginx/nginx.conf

添加:

user nobody nobody;#定义Nginx运行的用户和用户组
 
worker_processes 4;#nginx进程数,建议设置为等于CPU总核心数。
 
error_log logs/error.log info;#全局错误日志定义类型,[ debug | info | notice | warn | error | crit ]
 
worker_rlimit_nofile 1024;#一个nginx进程打开的最多文件描述符数目,所以建议与ulimit -n的值保持一致。
 
pid logs/nginx.pid;#进程文件
 
 
#工作模式及连接数上限
 
events {
 
use epoll;#参考事件模型,use [ kqueue | rtsig | epoll |/dev/poll | select | poll ]; epoll模型是Linux2.6以上版本内核中的高性能网络I/O模型
 
worker_connections 1024;#单个进程最大连接数(最大连接数=连接数*进程数)
 
}
 
 
#设定http服务器,利用它的反向代理功能提供负载均衡支持
 
http {
 
include mime.types;#文件扩展名与文件类型映射表
 
default_type application/octet-stream;#默认文件类型
 
#设定负载均衡的服务器列表
 
upstream tomcatxxxcom {
 
server 192.168.56.200:8080;
 
server 192.168.56.201:8080;
 
}
 
#设定日志格式
 
log_format www_xy_com ‘$remote_addr - $remote_user [$time_local] "$request" ‘
 
‘$status $body_bytes_sent "$http_referer" ‘
 
‘"$http_user_agent" "$http_x_forwarded_for"‘;
 
 
sendfile on;#开启高效文件传输模式,sendfile指令指定nginx是否调用sendfile函数来输出文件,对于普通应用设为 on,如果用来进行下载等应用磁盘IO重负载应用,可设置为off,以平衡磁盘与网络I/O处理速度,降低系统的负载。注意:如果图片显示不正常把这个改成off
 
keepalive_timeout 65;#长连接超时时间,单位是秒
 
 
#gzip on;
 
#设定虚拟主机,默认为监听80端口
 
server {
 
listen 80;
 
server_name tomcat.xxx.com;#域名可以有多个,用空格隔开
 
 
#charset koi8-r;
 
#设定本虚拟主机的访问日志
 
access_log /data/logs/access.log www_xy_com;
 
#对 "/" 启用反向代理
 
location /{
 
proxy_pass http://tomcatxxxcom;
 
proxy_set_header Host $host;
 
proxy_set_header X-Real-IP $remote_addr;
 
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
}
 
 
#error_page 500 502 503 504 /50x.html;
 
location =/50x.html {
 
root html;
 
}
 
}
 
}

3、创建logs所需要的文件夹/data /logs/

cd /
 
mkdir m 755 data
 
cd data
 
mkdir m 755 logs

 

4、启动tomcatnginx

/usr/local/tomcat/bin/startup.sh
 
/usr/local/nginx/sbin/nginx
 

5、修改hosts,加入

192.168.56.99 tomcat.xxx.com

 

6、访问http://redis.xxy.com


上面部分安装,我是用脚本装的。没按照上面来,使用上面配置时,启动nginx时,提示我"[emerg] 7458#0: unknown directive "<feff>user" in /usr/local/nginx/nginx.conf:1",

于是,我就直接拿着nginx.conf.default修改了,改改也能用。下面是我修改后的配置(v1下:/usr/local/nginx/nginx.conf):

worker_processes 1;
error_log logs/error.log info;
pid logs/nginx.pid;
 
events {
use epoll;
worker_connections 1024;
}
 
http {
include mime.types;
default_type application/octet-stream;
# 配置需要代理的服务器列表
upstream servers {
server v3:8080;
server v4:8080;
}
 
log_format main ‘$remote_addr - $remote_user [$time_local] "$request" ‘
‘$status $body_bytes_sent "$http_referer" ‘
‘"$http_user_agent" "$http_x_forwarded_for"‘;
 
access_log logs/access.log main;
 
sendfile on;
 
keepalive_timeout 65;
 
server {
listen 80;
server_name localhost;
 
access_log logs/host.access.log main;
 
location /{
#引用上面的配置
proxy_pass http://servers;
proxy_set_header Host $host;
proxy_set_header X-Real_IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
root html;
index index.html index.htm;
}
 
error_page 500502503504/50x.html;
location =/50x.html {
root html;
}
}
 
}
访问v1:80,使用rr策略自动轮巡v3,v4效果图:
    技术分享

技术分享


对了,上面那句v3,v4的提示语,是我echo一句话把tomcat默认的ROOT/index.jsp 给覆盖了,用作测试

技术分享
    1.首先使用keepalived + nginx + tomcat 实现


测试过程中发现,我收到将nginx kill之后keepalived居然ip不飘,检查了配置也没发现问题。唯一能让keepavlied执行notify.sh脚本时候就是在启动kp时,
会检查nginx是否存活,不存活则启动。 现在已经实现了

于是又加了个脚本单独去监控kp ,脚本名称为:monitor.sh,代码在下边
v1的keepalived脚本:
    
!ConfigurationFilefor keepalived
 
global_defs {
}
 
vrrp_script chk_nginx {
script "killall -0 nginx"
interval 1
weight -2
}
 
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 1
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
 
track_script {
chk_nginx
}
 
virtual_ipaddress {
192.168.33.181/24
}
 
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
 
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 2
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
 
track_script {
chk_nginx
}
 
virtual_ipaddress {
192.168.33.182/24
}
 
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}

v2的keepalived脚本:
!ConfigurationFilefor keepalived
 
global_defs {
}
 
vrrp_script chk_nginx {
script "killall -0 nginx"
interval 1
weight -2
}
 
 
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 1
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
 
track_script {
chk_nginx
}
 
virtual_ipaddress {
192.168.33.181/24
}
 
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
 
 
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 2
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
 
track_script {
chk_nginx
}
 
virtual_ipaddress {
192.168.33.182/24
}
 
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}

v1,v2共同脚本:
monitor.sh:
#!/bin/bash
while true;
do
A=`ps -ef|grep nginx |wc -l`
B=`ps -ef|grep keepalived |wc -l`
if[ $A -eq 1];then
echo ‘restart nginx!!!!‘
/usr/local/nginx/sbin/nginx
if[ $A -eq 1];then
if[ $B -gt 1];then
killall keepalived
service keepalived start &
fi
fi
fi
 
if[ $B -eq 1];then
service keepalived start &
fi
 
sleep 5
done

notify.sh
#!/bin/bash
# keepalived notify script
contact=‘root@localhost‘
notify(){
mailsubject="`hostname` to be $1: vip floating"
mailbody="`date ‘+%F %H:%M:%S‘`: vrrp transition, `hostname` changed to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
 
case"$1"in
master)
notify master
echo "变成主了,触发master!">>/root/k_tips.log
/usr/local/nginx/sbin/nginx
exit 0
;;
backup)
notify backup
echo "变成从了,触发backup事件!">>/root/k_tips.log
/usr/local/nginx/sbin/nginx
exit 0
;;
fault)
notify fault
echo "脑列了,触发fault事件!">>/root/k_tips.log
/usr/local/nginx/sbin/nginx -s stop
exit 0
;;
*)
echo ‘Usage: notify.sh {master|backup|fault}‘
exit 1
;;
esac


    2.使用keepalived + haproxy + tomcat 实现
        2.1 安装haproxy
            yum -y install haproxy
        2.2 修改配置
            
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the ‘-r‘ option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
# 上面说我们要把日志写到/var/log/haproxy.log需要修改以下两个地方:
1./etc/sysconfig/syslog , 这个我现在用的centos6.6已经没有了,取代它的是同目录下的rsyslog
vi /etc/sysconfig/rsyslog
这里对它做了以下更改:
修改前:SYSLOGD_OPTIONS="-c 5"
修改后:SYSLOGD_OPTIONS="-c 2 -r"
2. /etc/rsyslog.conf 修改日志存放位置
vi /etc/rsyslog.conf
我在local7.*下面加了一行,修改后是这样的:
# Save boot messages also to boot.loglocal7.* /var/log/boot.loglocal2.* /var/log/haproxy.log
好了,基本的配置可以了。
下面开始修改/etc/haproxy/haproxy.cfg

    
    3.使用keepalived + lvs + tomcat 实现

未完待续。。。

Lvs + Ngnix + Haproxy + Keepalived + Tomcat 实现三种HA软负载均衡和Tomcat Session共享

标签:lvs   haproxy   keepalived   tomcat   nginx   

原文地址:http://blog.csdn.net/u010811257/article/details/46432219

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!