keepalived高可用负载均衡器

标签：计算   .com   ann   backend   功能实现   ebs   可用性集   one   可用性   

一、集群相关概念简述

HA是High Available缩写，是双机集群系统简称，指高可用性集群，是保证业务连续性的有效解决方案，一般有两个或两个以上的节点，且分为活动节点及备用节点。

1、集群的分类：

• LB：负载均衡集群
  • lvs负载均衡
  • nginx反向代理
  • HAProxy
• HA：高可用集群
  • heartbeat
  • keepalived
  • redhat5 : cman + rgmanager , conga(WebGUI) --> RHCS（Cluster Suite）集群套件
  • redhat6 : cman + rgmanager , corosync + pacemaker
  • redhat7 : corosync + pacemaker
• HP：高性能集群

2、系统可用性的计算公式

A=MTBF/(MTBF+MTTR)

• A：高可用性，指标：95%, 99%, 99.5%, ...., 99.999%，99.9999%等
• MTBF：平均无故障时间
• MTTR：平均修复时间

二、keepalived相关概念

• vrrp协议：Virtual Redundant Routing Protocol 虚拟冗余路由协议
• Virtual Router：虚拟路由器
• VRID(0-255)：虚拟路由器标识
• master：主设备，当前工作的设备
• backup：备用设备
• priority：优先级，优先级越大优先工作，具体情况示工作方式决定
• VIP：虚拟IP地址，正真向客户服务的IP地址
• VMAC：虚拟MAC地址(00-00-5e-00-01-VRID)
• 抢占式：如果有优先级高的节点上线，则将此节点转为master
• 非抢占式：即使有优先级高的节点上线，在当前master工作无故障的情况运行抢占；等到此master故障后重新按优先级选举master
• 心跳：master将自己的心跳信息通知集群内的所有主机，证明自己正常工作
• 安全认证机制：
  • 无认证：任何主机都可成为集群内主机，强烈不推荐
  • 简单的字符认证：使用简单的密码进行认证
  • AH认证
• sync group：同步组，VIP和DIP配置到同一物理服务器上
• MULTICAST：组播，多播
• Failover：master故障，故障切换，故障转移
• Failback：故障节点重新上线，故障切回

三、keepalived

vrrp协议的软件实现，原生设计的目的为了高可用ipvs服务

• 基于vrrp协议完成地址流动；
• 为vip地址所在的节点生成ipvs规则（在配置文件中预先定义）；
• 为ipvs集群的各RS做健康状态检测；
• 基于脚本调用接口通过执行脚本完成脚本中定义的功能，进而影响集群事务；

组件：

1、安装

# yum install keepalived

主配置文件：/etc/keepalived/keepalived.conf

主程序文件：/usr/sbin/keepalived

启动服务：systemctl start keepalived

Unit File的环境配置文件：/etc/sysconfig/keepalived

2、配置文件参数详解

全局配置段：

global_defs {
   notification_email {  #发送通知email，收件人
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1  #邮件服务器地址
   smtp_connect_timeout 30  #超时时长
   router_id LVS_DEVEL  #路由器标识ID
   vrrp_skip_check_adv_addr  #跳过的检查地址
   vrrp_strict  #严格模式
   vrrp_garp_interval 0  #免费arp
   vrrp_gna_interval 0
}

虚拟路由实例段：

vrrp_instance <STRING> {
    state MASTER|BACKUP：#当前节点在此虚拟路由器上的初始状态；只能有一个是MASTER，余下的都应该为BACKUP；
    interface IFACE_NAME：#绑定为当前虚拟路由器使用的物理接口；
    virtual_router_id VRID：#当前虚拟路由器的惟一标识，范围是0-255；
    priority 100：#当前主机在此虚拟路径器中的优先级；范围1-254；
    advert_int 1：#vrrp通告的时间间隔；
    authentication {
        auth_type AH|PASS #pass为简单认证
        auth_pass <PASSWORD> #认证密码，8为密码
    }
    virtual_ipaddress {  #VIP配置
        <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
        192.168.200.17/24 dev eth1
        192.168.200.18/24 dev eth2 label eth2:1
    }
    track_interface {  #配置要监控的网络接口，一旦接口出现故障，则转为FAULT状态；
        eth0
        eth1
        ...
    }
    nopreempt：定义工作模式为非抢占模式；
    preempt_delay 300：抢占式模式下，节点上线后触发新选举操作的延迟时长；
    notify_master <STRING>|<QUOTED-STRING>：当前节点成为主节点时触发的脚本；
    notify_backup <STRING>|<QUOTED-STRING>：当前节点转为备节点时触发的脚本；
    notify_fault <STRING>|<QUOTED-STRING>：当前节点转为“失败”状态时触发的脚本；
    notify <STRING>|<QUOTED-STRING>：通用格式的通知触发机制，一个脚本可完成以上三种状态的转换时的通知；
}

虚拟服务器配置：

virtual_server IP port | virtual_server fwmark int {
    delay_loop <INT>：服务轮询的时间间隔；
    lb_algo rr|wrr|lc|wlc|lblc|sh|dh：定义调度方法；
    lb_kind NAT|DR|TUN：集群的类型；
    persistence_timeout <INT>：持久连接时长；
    protocol TCP：服务协议，仅支持TCP；
    sorry_server <IPADDR> <PORT>：备用服务器地址；
    real_server {
        weight <INT>
        notify_up <STRING>|<QUOTED-STRING>
        notify_down <STRING>|<QUOTED-STRING>
        HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { ... }：定义当前主机的健康状态检测方法；
    }
    HTTP_GET|SSL_GET：应用层检测
    HTTP_GET|SSL_GET {
        url {
            path <URL_PATH>：定义要监控的URL；
            status_code <INT>：判断上述检测机制为健康状态的响应码；
            digest <STRING>：判断上述检测机制为健康状态的响应的内容的校验码；
        }
        nb_get_retry <INT>：重试次数；
        delay_before_retry <INT>：重试之前的延迟时长；
        connect_ip <IP ADDRESS>：向当前RS的哪个IP地址发起健康状态检测请求
        connect_port <PORT>：向当前RS的哪个PORT发起健康状态检测请求
        bindto <IP ADDRESS>：发出健康状态检测请求时使用的源地址；
        bind_port <PORT>：发出健康状态检测请求时使用的源端口；
        connect_timeout <INTEGER>：连接请求的超时时长；
    }
     TCP_CHECK {
        connect_ip <IP ADDRESS>：向当前RS的哪个IP地址发起健康状态检测请求
        connect_port <PORT>：向当前RS的哪个PORT发起健康状态检测请求
        bindto <IP ADDRESS>：发出健康状态检测请求时使用的源地址；
        bind_port <PORT>：发出健康状态检测请求时使用的源端口；
        connect_timeout <INTEGER>：连接请求的超时时长；
    }
}

脚本定义：

vrrp_script <SCRIPT_NAME> {
    script ""  #定义执行脚本
    interval INT  #多长时间检测一次
    weight -INT  #如果脚本的返回值为假，则执行权重减N的操作
    rise 2  #检测2次为真，则上线
    fall 3  #检测3次为假，则下线
}
vrrp_instance VI_1 {
    track_script {  #在虚拟路由实例中调用此脚本
        SCRIPT_NAME_1
        SCRIPT_NAME_2
        ...
    }
}

四、lvs + keepalived的单主模型实现

环境：

• 各节点时间必须同步；
• 确保iptables及selinux的正确配置；
• 各节点之间可通过主机名互相通信（对KA并非必须），建议使用/etc/hosts文件实现；
• 确保各节点的用于集群服务的接口支持MULTICAST通信；D类：224-239；ip link set dev eth0 multicast off | on

node1配置：

[root@node1 ~]# yum install keepalied
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost #收件人
   }
   notification_email_from keepalived@localhoat #发件人
   smtp_server 127.0.0.1 #邮件服务器IP
   smtp_connect_timeout 30 #连接超时时长
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.111.111  #组播地址
   vrrp_iptables #禁止keepalived添加iptables规则
}
vrrp_instance VI_1 { #定义虚拟路由实例
    state MASTER #初始启动为主节点
    interface eth0 #IP属于的网卡
    virtual_router_id 51 #节点ID
    priority 100 #优先级
    advert_int 1 #每1秒检测一次
    authentication { #认证
        auth_type PASS #简单认证
        auth_pass fd57721a #认证密码
    }
    virtual_ipaddress { #vip绑定的网卡
        192.168.0.2/24 dev eth0
    }
}
virtual_server 192.168.0.2 80 { #ipvs规则定义
    delay_loop 2 #健康检测，2秒
    lb_algo rr #调度算法，轮询
    lb_kind DR #lvs模型，DR
    protocol TCP #tcp协议
    real_server 192.168.0.10 80 { #real-server配置
        weight 1 #权重为1
        HTTP_GET { #HTTP协议检测
            url {
              path / #检测主页
              status_code 200 #返回值为200为正常
            }
            connect_timeout 2 #超时时长
            nb_get_retry 3 #重连次数
            delay_before_retry 1 #重连间隔
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}
[root@node1 ~]# systemctl start keepalived.service

node2配置：

[root@node2 ~]# yum install keepalied
[root@node2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.111.111
   vrrp_iptables
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 98
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
        preempt_delay 300
}
virtual_server 192.168.0.2 80 {
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 192.168.0.10 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}
[root@node2 ~]# systemctl start keepalived.service

web1/web2/web3：配置脚本

#!/bin/bash
#
vip="192.168.0.2/32"
iface="lo"
case $1 in
start)
        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
        ip addr add $vip label $iface:0 broadcast ${vip%/*} dev $iface
        ip route add $vip dev $iface
        ;;
stop)
        ip addr flush dev $iface
        ip route flush dev $iface
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
        ;;
*)
        echo "Usage: `basename $0` start | stop" 1>&2
        ;;
esac

五、lvs + keepalived的双主模型实现

node1配置：

! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.111.111
   vrrp_iptables
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
}
vrrp_instance VI_2 {
    state BACKUP
    interface eth0
    virtual_router_id 52
    priority 98
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 4a9a407a
    }
    virtual_ipaddress {
        192.168.0.3/24 dev eth0
    }
}
virtual_server 192.168.0.2 80 {
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 192.168.0.10 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}
virtual_server 192.168.0.3 80 {
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.0.10 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}

node2配置：

! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.111.111
   vrrp_iptables
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 98
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface eth0
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 4a9a407a
    }
    virtual_ipaddress {
        192.168.0.3/24 dev eth0
    }
}
virtual_server 192.168.0.2 80 {
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.0.10 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}
virtual_server 192.168.0.3 80 {
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.0.10 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.11 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 192.168.0.12 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}

web1/web2/web3：配置脚本

#!/bin/bash
#
vip="192.168.0.2/32"
vip2="192.168.0.3/32"
iface="lo"
case $1 in
start)
        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
        ip addr add $vip label $iface:0 broadcast ${vip%/*} dev $iface
        ip addr add $vip2 label $iface:1 broadcast ${vip2%/*} dev $iface
        ip route add $vip dev $iface
        ip route add $vip2 dev $iface
        ;;
stop)
        ip addr flush dev $iface
        ip route flush dev $iface
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
        ;;
*)
        echo "Usage: `basename $0` start | stop" 1>&2
        ;;
esac

六、通知脚本的实现

[root@node1 ~]# vim /etc/keepalived/notify.sh
#!/bin/bash
#
contact='root@localhost'
notify() {
    local mailsubject="$(hostname) to be $1, vip floating"
    local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
    echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master) notify master;;
backup) notify backup;;
fault) notify fault;;
*) echo "Usage: $(basename $0) {master|backup|fault}"; exit 1;;
esac
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
    notify_master "/etc/keepalived/notify.sh master" #调用脚本发送通知邮件，当此节点转为master时
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

七、keepalived + haproxy 实现调度器的高可用

配置haproxy实现负载均衡的功能

[root@node1 ~]# vim /etc/haproxy/haproxy.cfg
frontend web *:80
    default_backend websrvs
backend websrvs
    balance roundrobin
    server srv1 192.168.0.10:80 check
    server srv2 192.168.0.11:80 check
    server srv3 192.168.0.12:80 check

配置keepalived实现高可用

[root@node1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.111.111
   vrrp_iptables
}
vrrp_script chk_haproxy {
        script "killall -0 haproxy"  #监控haproxy进程
        interval 1
        weight -5
        fall 2
        rise 1
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
    track_script {  #调用监控脚本
            chk_haproxy
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

八、利用脚本功能实现keepalived的维护模式

vrrp_script chk_down {
    script "/bin/bash -c '[[ -f /etc/keepalived/down ]]' && exit 1 || exit 0"  #在keepalived中要特别地指明作为bash的参数的运行
    interval 1
    weight -10
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass fd57721a
    }
    virtual_ipaddress {
        192.168.0.2/24 dev eth0
    }
    track_script {
        chk_down  #调用脚本
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}
测试：创建down文件后使得降优先级，从而使得VIP漂移到node2，进入维护模式
[root@node1 ~]# touch /etc/keepalived/down

九、编写Ansible角色批量部署keepalived + nginx 实现双主模型下反向代理器的高可用

以下操作全部在ansible主机上操作

1、基于秘钥通信：

[root@ansible ~]# vim cpkey.sh
#!/bin/bash
rpm -q expect &>/dev/null || yum -q -y install expect
[ ! -e ~/.ssh/id_rsa ] && ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa &>/dev/null
read -p "Host_ip_list: " ip_list_file
read -p "Username: " username
read -s -p "Password: " password
[ ! -e "$ip_list_file" ] && echo "$ip_list_file not exist." && exit
[ -z "$ip_list_file" -o -z "$username" -o -z "$password" ] && echo "input error!" && exit
localhost_ip=`hostname -I |cut -d' ' -f1`
expect <<EOF
set timeout 10
spawn ssh-copy-id -i /root/.ssh/id_rsa.pub $localhost_ip
expect {
"yes/no" { send "yes\n"; exp_continue}
"password" { send "$password\n"}
}
expect eof
EOF
while read ipaddr1; do
expect <<EOF
set timeout 10
spawn ssh ${username}@${ipaddr1} ':'
expect {
"yes/no" { send "yes\n"; exp_continue}
"password" { send "$password\n"}
}
expect eof
EOF
done < "$ip_list_file"
while read ipaddr2; do
expect <<EOF
set timeout 10
spawn scp -pr .ssh/ ${username}@${ipaddr2}:
expect {
"yes/no" { send "yes\n"; exp_continue}
"password" { send "$password\n"}
}
expect eof
EOF
done < "$ip_list_file"
[root@ansible ~]# vim iplist.txt
192.168.0.8
192.168.0.9
192.168.0.11
192.168.0.12
192.168.0.13
[root@ansible ~]# ./cpkey.sh
Host_ip_list: iplist.txt  #指定IP地址列表文件
Username: root
Password: ******

2、配置内部主机基于主机名通信：

[root@ansible ~]# vim /etc/hosts
192.168.0.8 node1
192.168.0.9 node2
192.168.0.10    dns
192.168.0.11    web1
192.168.0.12    web2
192.168.0.13    web3
192.168.0.13    ansible
[root@ansible ~]# yum install ansible -y （epel源）
[root@ansible ~]# vim /etc/ansible/hosts
[node]
192.168.0.8
192.168.0.9
[web]
192.168.0.11
192.168.0.12
192.168.0.13
[dns]
192.168.0.10
[root@ansible ~]# ansible all -m copy -a 'src=/etc/hosts dest=/etc/hosts backup=yes'

3、编写角色，实现web服务的部署

[root@ansible ~]# mkdir -p ansible/roles/web/{tasks,templates,files,handlers}
[root@ansible ~]# cd ansible/
[root@ansible ansible]# vim roles/web/tasks/install.yml
- name: install httpd
  yum: name=httpd state=present
[root@ansible ansible]# vim roles/web/tasks/copy.yml
- name: copy config file
  template: src=httpd.conf.j2 dest=/etc/httpd/conf/httpd.conf
  notify: restart service
- name: copy index.html
  template: src=index.html.j2 dest=/var/www/html/index.html owner=apache
  notify: restart service
[root@ansible ansible]# vim roles/web/tasks/start.yml
- name: start httpd
  service: name=httpd state=started
[root@ansible ansible]# vim roles/web/tasks/main.yml
- include: install.yml
- include: copy.yml
- include: start.yml
[root@ansible ansible]# yum install httpd -y
[root@ansible ansible]# cp /etc/httpd/conf/httpd.conf roles/web/templates/httpd.conf.j2
[root@ansible ansible]# vim roles/web/templates/httpd.conf.j2
ServerName {{ ansible_fqdn }}
[root@ansible ansible]# vim roles/web/templates/index.html.j2
{{ ansible_fqdn }} test page.
[root@ansible ansible]# vim roles/web/handlers/main.yml
- name: restart service
  service: name=httpd state=restarted
[root@ansible ansible]# vim web.yml
---
- hosts: web
  remote_user: root
  roles:
    - web
...
[root@ansible ansible]# ansible-playbook web.yml

4、编写角色，实现nginx反向代理服务的部署

[root@ansible ansible]# mkdir -p roles/nginx_proxy/{files,handlers,tasks,templates}
[root@ansible ansible]# vim roles/nginx_proxy/tasks/install.yml
- name: install nginx
  yum: name=nginx state=present
[root@ansible ansible]# vim roles/nginx_proxy/tasks/copy.yml
- name: copy config file
  template: src=nginx.conf.j2 dest=/etc/nginx/nginx.conf
  notify: restart service
[root@ansible ansible]# vim roles/nginx_proxy/tasks/start.yml
- name: start nginx
  service: name=nginx state=started
[root@ansible ansible]# vim roles/nginx_proxy/tasks/main.yml
- include: install.yml
- include: copy.yml
- include: start.yml
[root@ansible ansible]# yum install nginx -y
[root@ansible ansible]# cp /etc/nginx/nginx.conf roles/nginx_proxy/templates/nginx.conf.j2
[root@ansible ansible]# vim roles/nginx_proxy/templates/nginx.conf.j2
http {
    upstream websrvs {  #后端web服务器的IP地址
        server 192.168.0.11;
        server 192.168.0.12;
        server 192.168.0.13;
    }
    server {
        listen       80 default_server;
        server_name  _;
            root /usr/share/nginx/html;
        location / {
            proxy_pass http://websrvs;
        }
    }
}
[root@ansible ansible]# vim roles/nginx_proxy/handlers/main.yml
- name: restart service
  service: name=nginx state=restarted
[root@ansible ansible]# vim nginx_proxy.yml
---
- hosts: node
  remote_user: root
  roles:
    - nginx_proxy
...
[root@ansible ansible]# ansible-playbook nginx_proxy.yml

5、编写角色，利用keepalived实现nginx反向代理服务的高可用

[root@ansible ansible]# ansible 192.168.0.8 -m hostname -a 'name=node1'
[root@ansible ansible]# ansible 192.168.0.9 -m hostname -a 'name=node2'
[root@ansible ansible]# mkdir -p roles/keepalived/{files,handlers,tasks,templates,vars}
[root@ansible ansible]# vim roles/keepalived/tasks/install.yml  #安装剧本
- name: install keepalived
  yum: name=keepalived state=present
[root@ansible ansible]# vim roles/keepalived/tasks/copy.yml  #复制配置文件剧本
- name: copy configure file
  template: src=keepalived.conf.j2 dest=/etc/keepalived/keepalived.conf
  notify: restart service
  when: ansible_fqdn == "node1"  #选择性复制，将第一套配置文件复制到node1上
- name: copy configure file2
  template: src=keepalived.conf2.j2 dest=/etc/keepalived/keepalived.conf
  notify: restart service
  when: ansible_fqdn == "node2"  #将第二套配置文件复制到node2上
[root@ansible ansible]# vim roles/keepalived/tasks/start.yml  #启动服务
- name: start keepalived
  service: name=keepalived state=started
[root@ansible ansible]# vim roles/keepalived/tasks/main.yml
- include: install.yml
- include: copy.yml
- include: start.yml
[root@ansible ansible]# vim roles/keepalived/vars/main.yml  #自定义变量
kepd_vrrp_mcast_group4: "224.0.111.222"  #组播地址
kepd_interface_1: "eth0"
kepd_virtual_router_id_1: "51"  #虚拟路由标识ID
kepd_priority_1: "100"  #优先级
kepd_auth_pass_1: "fd57721a"  #简单认证密码，8位
kepd_virtual_ipaddress_1: "192.168.0.2/24"  #VIP地址，此处应该为公网地址
kepd_interface_2: "eth0"
kepd_virtual_router_id_2: "52"
kepd_priority_2: "98"
kepd_auth_pass_2: "41af6acc"
kepd_virtual_ipaddress_2: "192.168.0.3/24"
[root@ansible ansible]# yum install keepalived -y
[root@ansible ansible]# cp /etc/keepalived/keepalived.conf roles/keepalived/templates/keepalived.conf.j2
[root@ansible ansible]# vim roles/keepalived/templates/keepalived.conf.j2  #编辑配置文件模板
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 {{ kepd_vrrp_mcast_group4 }}
   vrrp_iptables
}
vrrp_script chk_nginx {
        script "killall -0 nginx"
        interval 1
        weight -5
        fall 2
        rise 1
}
vrrp_instance VI_1 {
    state MASTER
    interface {{ kepd_interface_1 }}
    virtual_router_id {{ kepd_virtual_router_id_1 }}
    priority {{ kepd_priority_1 }}
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass {{ kepd_auth_pass_1 }}
    }
    virtual_ipaddress {
        {{ kepd_virtual_ipaddress_1 }}
    }
    track_script {
            chk_nginx
    }
}
vrrp_instance VI_2 {
    state BACKUP
    interface {{ kepd_interface_2 }}
    virtual_router_id {{ kepd_virtual_router_id_2 }}
    priority {{ kepd_priority_2 }}
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass {{ kepd_auth_pass_2 }}
    }
    virtual_ipaddress {
        {{ kepd_virtual_ipaddress_2 }}
    }
    track_script {
            chk_nginx
    }
}
[root@ansible ansible]# cp roles/keepalived/templates/keepalived.conf.j2 roles/keepalived/templates/keepalived.conf2.j2
[root@ansible ansible]# vim roles/keepalived/templates/keepalived.conf2.j2  #编写第二套配置文件，和第一套不同的只有state和priority参数需要改
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhoat
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 {{ kepd_vrrp_mcast_group4 }}
   vrrp_iptables
}
vrrp_script chk_nginx {
        script "killall -0 nginx"
        interval 1
        weight -5
        fall 2
        rise 1
}
vrrp_instance VI_1 {
    state BACKUP
    interface {{ kepd_interface_1 }}
    virtual_router_id {{ kepd_virtual_router_id_1 }}
    priority {{ kepd_priority_2 }}
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass {{ kepd_auth_pass_1 }}
    }
    virtual_ipaddress {
        {{ kepd_virtual_ipaddress_1 }}
    }
    track_script {
            chk_nginx
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface {{ kepd_interface_2 }}
    virtual_router_id {{ kepd_virtual_router_id_2 }}
    priority {{ kepd_priority_1 }}
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass {{ kepd_auth_pass_2 }}
    }
    virtual_ipaddress {
        {{ kepd_virtual_ipaddress_2 }}
    }
    track_script {
            chk_nginx
    }
}
[root@ansible ansible]# vim roles/keepalived/handlers/main.yml  #实现配置文件变动时触发重启服务操作
- name: restart service
  service: name=keepalived state=restarted
[root@ansible ansible]# vim keepalived.yml
---
- hosts: node
  remote_user: root
  roles:
    - keepalived
...
[root@ansible ansible]# ansible-playbook keepalived.yml

6、配置DNS

[root@dns ~]# yum install bind -y
[root@dns ~]# vim /etc/named.conf
[root@dns ~]# vim /etc/named.conf  #将以下参数注释
//listen-on port 53 { 127.0.0.1; };
//allow-query     { localhost; };
[root@dns ~]# vim /etc/named.rfc1912.zones
zone "dongfei.tech" {
    type master;
    file "dongfei.tech.zone";
};
[root@dns ~]# vim /var/named/dongfei.tech.zone
$TTL 1D
@ IN    SOA dns1.dongfei.tech. admin.dongfei.tech. ( 1 1D 1H 1W 3H )
        NS  dns1
dns1    A   192.168.0.10
www     A   192.168.0.2
www     A   192.168.0.3
[root@dns ~]# named-checkconf 
[root@dns ~]# named-checkzone "dongfei.tech" /var/named/dongfei.tech.zone
OK
[root@dns ~]# systemctl start named
[root@dns ~]# dig www.dongfei.tech @192.168.0.10
;; QUESTION SECTION:
;www.dongfei.tech.              IN      A
;; ANSWER SECTION:
www.dongfei.tech.       86400   IN      A       192.168.0.3
www.dongfei.tech.       86400   IN      A       192.168.0.2
;; AUTHORITY SECTION:
dongfei.tech.           86400   IN      NS      dns1.dongfei.tech.
;; ADDITIONAL SECTION:
dns1.dongfei.tech.      86400   IN      A       192.168.0.10
;; SERVER: 192.168.0.10#53(192.168.0.10)

7、客户端模拟测试

[root@client ~]# vim /etc/resolv.conf
nameserver 192.168.0.10
[root@client ~]# for i in {1..3}; do curl www.dongfei.tech; done
web2 test page.
web2 test page.
web3 test page.
将node1停机再测试
[root@client ~]# for i in {1..3}; do curl www.dongfei.tech; done
web2 test page.
web3 test page.
web1 test page.
现在已经实现了负载均衡的高可用！

附ansible角色下载：https://files.cnblogs.com/files/L-dongf/web_lb_cluster.tar.gz

感谢阅读~

keepalived高可用负载均衡器

标签：计算   .com   ann   backend   功能实现   ebs   可用性集   one   可用性   

原文地址：https://www.cnblogs.com/L-dongf/p/9300875.html