码迷，mamicode.com
首页 > 其他好文 > 详细

多puppetmaster，多ca，keepalived+haproxy（nginx）puppet集群搭建

时间：2015-03-05 16:14:49      阅读：260      评论：0      收藏：0      [点我收藏+]

标签：

多puppetmaster，多ca，keepalived+haproxy（nginx）puppet集群搭建

 

一、服务器详情

192.168.122.111 pm01.jq.com pm01 #（puppetmaster服务器）

192.168.122.112 pm02.jq.com pm02 #（puppetmaster服务器）

192.168.122.121 ag01.jq.com ag01 #（puppet agent服务器）

192.168.122.122 ag02.jq.com ag02 #（puppet agent服务器）

192.168.122.131 ca01.jq.com ca01 #（puppet ca服务器）

192.168.122.132 ca02.jq.com ca02 #（puppet ca服务器）

192.168.122.141 lvs01.jq.com lvs01 #（puppet 负载服务器）

192.168.122.142 lvs02.jq.com lvs02 #（puppet 负载服务器）

#vip暂时用于测试，使用ip addr的方式绑定，后续用高可用软件实现bind

192.168.122.130 pc.jq.com pc #（ca服务器的vip，前期绑定在ca01服务器上）

192.168.122.115 lvs.jq.com lvs #（负载服务器的vip，前期绑定在puppetmaster上，后期需要绑定在lvs服务器上）

 

二、CA服务器部署

CA服务器单独用于签署和撤销证书，当puppetCA服务不可用时，新的客户端将不能获得证书，从而会影响使用，而已签发证书的客户端缺不受影响。因此将CA进行独立队架构，这对容错性而言是非常有必要的。

 

2.1 安装软件包

[root@ca01 ~]# groupadd -g 3000 puppet

[root@ca01 ~]# useradd -u 3000 -g 3000 puppet

[root@ca01 ~]# yum install puppet puppet-server –y

2.2 bind vip

绑定ca的vip 192.168.122.130到ca01服务器

[root@ca01 ~]#ip addr add 192.168.122.130/24 dev eth0

 

2.3 生成证书

使用puppet cert命令生成CA服务器与服务器域名证书。生成pc.jq.com和lvs.jq.com两个域名的授权证书文件,前面是证书别名，后面是证书名称。

[root@ca01 ssl]# puppet  cert --generate --dns_alt_names pc pc.jq.com

[root@ca01 ssl]# puppet  cert --generate --dns_alt_names lvs lvs.jq.com [root@ca01 ssl]# puppet cert --list --all

+ "lvs.jq.com" (SHA256) D6:5B:51:D6:6E:35:61:A4:45:D8:37:17:5B:85:A1:1B:34:BB:2F:D7:48:E8:44:57:B7:1D:42:8E:11:18:81:34 (alt names: "DNS:lvs", "DNS:lvs.jq.com")

+ "pc.jq.com"  (SHA256) A7:71:E1:46:1E:F0:F1:70:72:E3:B5:16:03:91:17:6D:68:5B:55:39:B6:79:6B:30:DD:41:ED:10:21:27:2A:33 (alt names: "DNS:pc", "DNS:pc.jq.com")

2.4 配置puppet.conf，添加标签[master]

[root@ca01 ~]# cat /etc/puppet/puppet.conf  | grep -v "#"

[main]

    logdir = /var/log/puppet

    rundir = /var/run/puppet

    ssldir = $vardir/ssl

    pluginsync = false

[agent]

    classfile = $vardir/classes.txt

    localconfig = $vardir/localconfig

    server = lvs.jq.com

    ca_server = pc.jq.com

    environment = jqprd

[master]

    confdir = /etc/puppet

    certname = pc.jq.com

ca = true #开启CA认证

 

2.5 启动puppetmaster，CA部署完成

[root@ca01 ~]# /etc/init.d/puppetmaster start

[root@ca01 ~]# chkconfig puppetmaster on

 

2.6 ca02服务部署

ca02跟ca01的部署方式完全一致，证书是从ca01 拷贝过来的。直接copy /var/lib/ssl/puppet目录

 

三、PuppetMaster服务器部署

PuppetMaster服务器部署可采用默认的WebRick方式，也可以采用apache+passenger或nginx+passenger方式。

3.1 WebRick方式：

3.1.1 安装软件包

[root@pm01 ~]# groupadd -g 3000 puppet

[root@pm01 ~]# useradd -u 3000 -g 3000 puppet

[root@pm01 ~]# yum install puppet puppet-server -y

3.1.2 设置hosts文件

[root@pm01 ~]# vim /etc/hosts

192.168.122.111 pm01.jq.com pm01

192.168.122.112 pm02.jq.com pm02

192.168.122.121 ag01.jq.com ag01

192.168.122.122 ag02.jq.com ag02

192.168.122.131 ca01.jq.com ca01

192.168.122.132 ca02.jq.com ca02

192.168.122.141 lvs01.jq.com lvs01

192.168.122.142 lvs02.jq.com lvs02

192.168.122.130 pc.jq.com pc

192.168.122.115 lvs.jq.com lvs

3.1.3 bind master vip

绑定LVS的vip 192.168.122.115到pm01服务器，测试用，在没有负载之前，绑定在master上。

ip addr add 192.168.122.115/24 dev eth0

3.1.4 创建证书目录

[root@pm01 ~]# mkdir /var/lib/puppet/ssl/{certs,ca,private_keys} –p

 

3.1.5 将puppetca上生成的puppetmaster公钥、私钥和根证书复制到pm01

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/signed/lvs.jq.com.pem /var/lib/puppet/ssl/certs/lvs.jq.com.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/ca_crt.pem /var/lib/puppet/ssl/certs/ca.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/private_keys/lvs.jq.com.pem /var/lib/puppet/ssl/private_keys/lvs.jq.com.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/ca_crl.pem /var/lib/puppet/ssl/ca/ca_crl.pem

3.1.6 配置puppet.conf，添加标签[master]，关闭ca

 

[root@pm01 ~]# grep -v "#" /etc/puppet/puppet.conf

[main]

    logdir = /var/log/puppet

    rundir = /var/run/puppet

    ssldir = $vardir/ssl

    privatekeydir = $ssldir/private_keys { group = service } 

    hostprivkey = $privatekeydir/$certname.pem { mode = 640 }

 

[agent]

    classfile = $vardir/classes.txt

    localconfig = $vardir/localconfig

    server = lvs.jq.com #puppetmaster域名，应该与之前手动生成的证书匹配

    ca_server = pc.jq.com #ca证书服务器

 

[master]

    certname = lvs.jq.com #puppetmaster的域名，应该与之前手动生成的证书匹配

    ca = false #关闭ca验证

 

3.1.7 启动puppetmaster服务，puppetmaster部署完成

[root@pm01 ssl]# /etc/init.d/puppetmaster restart

 

3.1.8 运行puppet命令进行本地证书申请

[root@pm01 ssl]# puppet  agent -t

Info: Creating a new SSL key for pm01.jq.com

Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml

Info: Creating a new SSL certificate request for pm01.jq.com

Info: Certificate Request fingerprint (SHA256): 2C:09:32:E1:13:CA:0F:44:3B:93:4B:0F:0E:2D:46:19:3A:37:E1:47:C7:D3:E8:2C:A6:83:44:B3:D3:94:63:D6

Exiting; no certificate found and waitforcert is disabled

 

3.1.9 登录puppetca进行证书签发

[root@ca01 ~]# puppet cert --sign pm01.jq.com

Notice: Signed certificate request for pm01.jq.com

Notice: Removing file Puppet::SSL::CertificateRequest pm01.jq.com at ‘/var/lib/puppet/ssl/ca/requests/pm01.jq.com.pem‘

3.1.10 再次运行puppet命令进行测试连通性

[root@pm01 ssl]# puppet  agent -t

Info: Caching certificate for pm01.jq.com

Info: Caching certificate_revocation_list for ca

Info: Caching certificate for pm01.jq.com

Info: Retrieving pluginfacts

Info: Retrieving plugin

Info: Caching catalog for pm01.jq.com

Info: Applying configuration version ‘1425526708‘

Notice: Finished catalog run in 0.17 seconds

3.1.11 在kspupt-ca上申请本地证书

[root@ca01 ~]# vim /etc/puppet/puppet.conf

[agent]

    server    = lvs.jq.com

    ca_server = pc.jq.com

[root@ca01 ~]# puppet agent -t

[root@ca01 ~]# puppet cert --sign ca01.jq.com

[root@ca01 ~]# puppet agent –t

 

3.2 Nginx+Passenger方式：

注：可参考 http://kisspuppet.com/2014/10/20/puppet_learning_ext4/

3.2.1、配置nginx

[root@pm01 ssl]# cat /usr/local/nginx/conf/vhosts/passenger.conf

    server {

        listen 8140                ssl;

    server_name                puppetmaster;

    passenger_enabled          on;

    passenger_set_cgi_param    HTTP_X_CLIENT_DN $ssl_client_s_dn;

    passenger_set_cgi_param    HTTP_X_CLIENT_VERIFY $ssl_client_verify;

    proxy_buffer_size 4000k;

    proxy_buffering on;

    proxy_buffers 32 1280k;

    proxy_busy_buffers_size 17680k;

    client_max_body_size 10m;

    client_body_buffer_size 4096k;

    access_log /var/log/nginx/puppet_access.log;

    error_log /var/log/nginx/puppet_error.log;

    root /etc/puppet/rack/public;

    ssl off;

    ssl_session_timeout 5m;

    ssl_certificate /var/lib/puppet/ssl/certs/lvs.jq.com.pem;

    ssl_certificate_key /var/lib/puppet/ssl/private_keys/lvs.jq.com.pem;

    ssl_client_certificate /var/lib/puppet/ssl/certs/ca.pem;

    ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;

    ssl_verify_client optional;

    ssl_ciphers SSLv2:-LOW:-EXPORT:RC4+RSA;

    ssl_prefer_server_ciphers on;

    ssl_verify_depth 1;

    ssl_session_cache shared:SSL:128m;

    # File sections

    location /production/file_content/files/ {

      types { }

      default_type application/x-raw;

      alias /etc/puppet/files/;

  }

 }

 

3.2.2、配置puppet.conf

[root@pm01 ssl]# grep -v "#" /etc/puppet/puppet.conf

[main]

    logdir = /var/log/puppet

    rundir = /var/run/puppet

    ssldir = $vardir/ssl

    privatekeydir = $ssldir/private_keys { group = service } 

    hostprivkey = $privatekeydir/$certname.pem { mode = 640 }

 

[agent]

    classfile = $vardir/classes.txt

    localconfig = $vardir/localconfig

    server = lvs.jq.com

    ca_server = pc.jq.com

 

[master]

    certname = lvs.jq.com

    ca = false

    ssl_client_verify_header = HTTP_X_CLIENT_VERIFY

ssl_client_header = HTTP_X_CLIENT_DN

 

3.3 master02服务器部署

master02的部署与master01的完全一致，包括拷贝证书部分

 

4 Puppet LB负载均衡器部署

4.1 puppet认证建立

4.1.1、安装软件包

[root@lvs01 ~]# groupadd -g 3000 puppet

[root@lvs01 ~]# useradd -u 3000 -g 3000 puppet

[root@lvs01 ~]# yum install puppet

4.1.2、编辑hosts文件

[root@lvs01 ~]# vim /etc/hosts

192.168.122.111 pm01.jq.com pm01

192.168.122.112 pm02.jq.com pm02

192.168.122.121 ag01.jq.com ag01

192.168.122.122 ag02.jq.com ag02

192.168.122.131 ca01.jq.com ca01

192.168.122.132 ca02.jq.com ca02

192.168.122.141 lvs01.jq.com lvs01

192.168.122.142 lvs02.jq.com lvs02

192.168.122.130 pc.jq.com pc

192.168.122.115 lvs.jq.com lvs

 

4.1.3、创建证书目录

[root@lvs01 ~]# mkdir /var/lib/puppet/ssl/{certs,ca,private_keys} –p

 

4.1.4、将ca上生成的puppetmaster公钥、私钥和根证书复制到lvs01

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/signed/lvs.jq.com.pem /var/lib/puppet/ssl/certs/lvs.jq.com.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/ca_crt.pem /var/lib/puppet/ssl/certs/ca.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/private_keys/lvs.jq.com.pem /var/lib/puppet/ssl/private_keys/lvs.jq.com.pem

scp -r root@192.168.122.130:/var/lib/puppet/ssl/ca/ca_crl.pem /var/lib/puppet/ssl/ca/ca_crl.pem

 

4.1.5、配置puppet.conf，修改标签[agent]，增加server和ca_server字段

[root@lvs01 ~]# vim /etc/puppet/puppet.conf

[agent]

    server      = lvs.jq.com

    ca_server   = pc.jq.com

4.1.6、运行puppet命令进行本地证书申请

[root@lvs01 ~]# puppet  agent -t

4.1.7、登录ca进行证书签发

[root@ca01 ~]# puppet  cert --sign lvs01.jq.com

4.1.8、再次运行puppet命令进行测试连通性

[root@lvs01 ~]# puppet agent -t

Info: Caching certificate for lvs01.jq.com

Info: Caching certificate_revocation_list for ca

Info: Caching certificate for lvs01.jq.com

Info: Loading facts

Info: Caching catalog for lvs01.jq.com

Info: Applying configuration version ‘1425527450‘

Notice: Finished catalog run in 0.24 seconds

 

4.2 安装并配置nginx负载均衡器

4.2.1、安装nginx软件

[root@lvs01 ~]# groupadd -g 3001 nginx

[root@lvs01 ~]# useradd -u 3001 -g 3001 nginx

[root@lvs01 ~]# yum install nginx

4.2.2、临时设置VIP地址（后面通过高可用软件代替）

[root@lvs01 ~]# ip addr add 192.168.122.115/24 dev eth0

此处请将之前bind到pm01的vip取消

4.2.3、配置nginx虚拟主机，添加upstrem

[root@lvs01 ~]# cat /etc/nginx/conf.d/puppetmaster.conf

upstream puppet-master {

   server 192.168.122.111:8140;

   server 192.168.122.112:8140;

}

server {

   listen         8140 ssl;

   server_name    puppetmaster;

   access_log     /var/log/nginx/puppet_access.log;

   error_log      /var/log/nginx/puppet_error.log;

   ssl_protocols SSLv3 TLSv1;

   ssl_ciphers ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP;

   proxy_set_header             X-SSL-Subject  $ssl_client_s_dn;

   proxy_set_header             X-Client-DN  $ssl_client_s_dn;

   proxy_set_header             X-Client-Verify  $ssl_client_verify;

   client_max_body_size 100m;

   client_body_buffer_size 1024k;

   proxy_buffer_size 100m;

   proxy_buffers 8 100m;

   proxy_busy_buffers_size 100m;

   proxy_temp_file_write_size 100m;

   proxy_read_timeout 500;

 

   ssl                     on;

   ssl_session_timeout     5m;

   ssl_certificate         /var/lib/puppet/ssl/certs/lvs.jq.com.pem;

   ssl_certificate_key     /var/lib/puppet/ssl/private_keys/lvs.jq.com.pem;

   ssl_client_certificate  /var/lib/puppet/ssl/certs/ca.pem;

   ssl_crl                 /var/lib/puppet/ssl/ca/ca_crl.pem;

   ssl_verify_client       optional;

   ssl_prefer_server_ciphers  on;

   ssl_verify_depth           1;

   ssl_session_cache          shared:SSL:128m;

   location / {

         proxy_redirect    off;

         proxy_pass        https://puppet-master;

  }

}

 

4.2.4、编辑hosts文件，puppetmaster解析指向VIP

[root@lvs01 ~]# vim /etc/hosts

192.168.122.111 pm01.jq.com pm01

192.168.122.112 pm02.jq.com pm02

192.168.122.121 ag01.jq.com ag01

192.168.122.122 ag02.jq.com ag02

192.168.122.131 ca01.jq.com ca01

192.168.122.132 ca02.jq.com ca02

192.168.122.141 lvs01.jq.com lvs01

192.168.122.142 lvs02.jq.com lvs02

192.168.122.130 pc.jq.com pc

192.168.122.115 lvs.jq.com lvs

4.2.5、修改ca01和pm01的hosts文件puppetmaster解析

[root@kspupt-ca1 ~]# vim /etc/hosts

192.168.122.111 pm01.jq.com pm01

192.168.122.112 pm02.jq.com pm02

192.168.122.121 ag01.jq.com ag01

192.168.122.122 ag02.jq.com ag02

192.168.122.131 ca01.jq.com ca01

192.168.122.132 ca02.jq.com ca02

192.168.122.141 lvs01.jq.com lvs01

192.168.122.142 lvs02.jq.com lvs02

192.168.122.130 pc.jq.com pc

192.168.122.115 lvs.jq.com lvs

 

[root@pm01 ~]# vim /etc/hosts

192.168.122.111 pm01.jq.com pm01

192.168.122.112 pm02.jq.com pm02

192.168.122.121 ag01.jq.com ag01

192.168.122.122 ag02.jq.com ag02

192.168.122.131 ca01.jq.com ca01

192.168.122.132 ca02.jq.com ca02

192.168.122.141 lvs01.jq.com lvs01

192.168.122.142 lvs02.jq.com lvs02

192.168.122.130 pc.jq.com pc

192.168.122.115 lvs.jq.com lvs

4.2.6、启动nginx服务器

[root@lvs01 ~]# /etc/init.d/nginx start

4.2.7、再次运行puppet命令进行测试连通性

[root@kspupt-ca1 ~]# puppet  agent -t

[root@pm01 ~]# puppet  agent -t

[root@lvs01 ~]# puppet  agent -t

[root@pm01 ~]# tailf  /var/log/nginx/puppet_access.log

[root@lvs01 ~]# tailf /var/log/nginx/puppet_access.log

 

4.3 安装配置Haproxy负载均衡

安装haproxy和keepalived过程略去，网上很多教程

 

[root@lvs01 keepalived]# cat /etc/haproxy/haproxy.cfg

global

        maxconn         40000

        ulimit-n        500000

        log             127.0.0.1 local0

        uid             99

        gid             99

        chroot          /tmp

#       nbproc          4

        daemon

 

defaults

        log     global

        retries 2

        option redispatch

        option dontlognull

        balance roundrobin

        timeout connect 30000ms

        timeout client 30000ms

        timeout server 30000ms

        timeout check 2000

 

listen admin_stats

    bind 0.0.0.0:8080

    mode http

    stats refresh 5s

    stats enable

    stats hide-version

    stats realm Haproxy\ Statistics

    stats uri /haproxy

    stats auth admin:password

 

listen puppetmaster *:8140

    mode tcp

    option ssl-hello-chk

#    option tcplog

    #balance source

#    balance roundrobin

    balance source

    server pm01 pm01.jq.com:8140 check inter 2000 fall 3

server pm02 pm02.jq.com:8140 check inter 2000 fall 3

 

4.4 配置keepalived，取消vip 通过ip addr 绑定

Keepalived的备机配置略去，网上也可以搜索到，需要修改的地方很少。

[root@lvs01 ~]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

 

global_defs {

   notification_email {

     test@gmail.com

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_script chk_http_port {

   script "/etc/keepalived/check_haproxy.sh"

   interval 2

   weight 2

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 51

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

track_script {

    chk_http_port

}

 

    virtual_ipaddress {

        192.168.122.115 #此处为负载的VIP，配置keepalived之后，切记去掉之前ip addr的绑定

    }

}

4.4.1 Keepalived监控haproxy脚本

[root@lvs01 ~]# cat /etc/keepalived/check_haproxy.sh

#!/bin/bash

. /etc/profile

A=`ps -C haproxy --no-header |wc -l`

if [ $A -eq 0 ];then

        /etc/init.d/haproxy start

sleep 3

if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then

/etc/init.d/keepalived stop

fi

fi

 

4.5 Lvs02服务器部署

Lvs02的配置与lvs01的配置完全一致，将此服务器作为lvs01的备机，包括keepalived+haproxy。

 

几乎照搬了http://kisspuppet.com/2014/10/21/puppet_learning_ext6/ 的文章，非常感谢kisspuppet！

 

多puppetmaster，多ca，keepalived+haproxy（nginx）puppet集群搭建

标签：

原文地址：http://www.cnblogs.com/krainbow/p/4315740.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！