码迷,mamicode.com
首页 > 其他好文 > 详细

ansible playbook部署ELK集群系统

时间:2017-12-16 17:10:45      阅读:386      评论:0      收藏:0      [点我收藏+]

标签:tor   htpasswd   密码   nginx配置   /etc/   脚本   pos   root   根据   

一、介绍

总共4台机器,分别为

192.168.1.99

192.168.1.100

192.168.1.210

192.168.1.211

服务所在机器为:

redis:192.168.1.211

logstash: 192.168.210 192.168.1.211

elasticsearch: 分别布置在四台(211不为主、不存储数据)

kibana:192.168.1.211

nginx:192.168.1.211

 

二、elk集群架构图

技术分享图片

 

三、为每一个软件或集群创建一个单独的角色。

cd /etc/ansible
mkdir -pv elk/roles/{jdk,redis,logstash,es,es_head,kibana,nginx}/{tasks,files,templates,meta,handlers,vars}

技术分享图片

 

四、配置hosts

在/etc/ansible/hosts添加如下内容:

[jdk_servers]
192.168.1.99
192.168.1.100
192.168.1.210
192.168.1.211

[redis_servers]
192.168.1.211

[es_servers]
192.168.1.99  node_name=node-99  node_master=true node_data=true network_host=192.168.1.99
192.168.1.100 node_name=node-100 node_master=true node_data=true network_host=192.168.1.100
192.168.1.210 node_name=node-210 node_master=true node_data=true network_host=192.168.1.210
192.168.1.211 node_name=node-211 node_master=true node_data=true network_host=192.168.1.211

[es_plugin_head]
192.168.1.211

[logstash_servers]
192.168.1.210
192.168.1.211

[kibana_servers]
192.168.1.211

[nginx_servers]
192.168.1.211

PS:es_servers组里面,后面定义了变量,因为es配置文件中,会根据服务器的不同,对应的配置也不同,在这里定义,可以直接在tasks里面去引用

 

五、JDK roles

1.把jdk-8u91-linux-x64.tar.gz和Java环境变量脚本传到files目录下

2.编写vars

[root@master jdk]# cat vars/main.yml 
jdk_package_name: jdk-8u91-linux-x64.tar.gz
env_dir: /etc/profile.d

3.编写tasks

[root@master jdk]# cat tasks/main.yml 
- name: copy jdk software
  copy: src={{ jdk_package_name }} dest=/tmp/ owner=root group=root
- name: uncompression jdk software
  shell: tar -zxf /tmp/{{ jdk_package_name }} -C /usr/local/
- name: copy jdk env 
  copy: src=java.sh dest={{ env_dir }}/ owner=root group=root mode=0644
- name: enforce env
  shell: source {{ env_dir }}/java.sh
- name: delete jdk compressin files
  shell: rm -f /tmp/{{ jdk_package_name }}

 4.使用角色

在roles同级目录,创建一个jdk.yml文件,里面定义好你的playbook。

[root@master elk]# cat jdk.yml 
- hosts: 127.0.0.1
  remote_user: root
  roles:
    - jdk

5. 运行playbook安装jdk

[root@master elk]# ansible-playbook jdk.yml

 

六、redis roles

1.把redis.tar.gz传到files目录下,redis配置文件和启动脚本传到templates目录下

2.编写vars

[root@master redis]# cat vars/main.yml 
redis_package_file: redis.tar.gz
redis_dir: /usr/local/redis
exec_dir: /etc/init.d

3.编写tasks

[root@master redis]# cat tasks/main.yml 
- name: copy redis software
  copy: src={{ redis_package_file }} dest=/tmp/ owner=root group=root
- name: uncompression redis software
  shell: tar zxf /tmp/{{ redis_package_file }} -C /usr/local
- name: copy redis start script
  template: src=redis dest={{ exec_dir }}/ owner=root group=root mode=0755
- name: copy redis config
  template: src=redis.conf dest={{ redis_dir }}/etc/ owner=redis group=redis 
- name: create redis user
  user: name=redis shell=/sbin/nologin
- name: change owner and group
  file: path={{ redis_dir }} owner=redis group=redis #recurse=yes
- name: mkdir directory for redis data
  file: dest=/data/redis mode=0755 state=directory owner=redis group=redis
- name: start redis service
  shell: /etc/init.d/redis start
- name: add boot start redis start
  shell: chkconfig --level 345 redis on
- name: delete redis compression files
  shell: rm -f /tmp/{{ redis_package_file }}

4.使用角色

在roles同级目录,创建一个redis.yml文件,里面定义好你的playbook

[root@master elk]# cat redis.yml 
- hosts: redis_servers
  remote_user: root
  roles:
    - redis

5. 运行playbook安装redis

[root@master elk]# ansible-playbook redis.yml

 

七、logstash roles

1.把logstash.tar.gz传到files目录下,logstash自定义配置文件和启动脚本传到templates目录下

2.编写vars

[root@master logstash]# cat vars/main.yml 
logstash_package_file: logstash.tar.gz
logstash_dir: /usr/local/logstash

3.编写tasks

[root@master logstash]# cat tasks/main.yml 
- name: copy logstash software
  copy: src={{ logstash_package_file }} dest=/tmp/ owner=root group=root
- name: uncompression logstash software
  shell: tar zxf /tmp/{{ logstash_package_file }} -C /usr/local
- name: mkdir config files dir
  file: dest={{ logstash_dir }}/conf.d mode=0755 state=directory owner=es group=es
- name: copy logstash start script
  template: src=start.sh dest={{ logstash_dir }}/ owner=es group=es mode=0755
- name: copy logstash config
  template: src=visit.conf dest={{ logstash_dir }}/conf.d/ owner=es group=es
- name: create es user
  user: name=es 
- name: change owner and group
  file: path={{ logstash_dir }} owner=es group=es
- name: start logstash service
  shell: su - es -c "{{ logstash_dir }}/start.sh"
- name: delete logstash compression files
  shell: rm -f /tmp/{{ logstash_package_file }}

4.使用角色

在roles同级目录,创建一个logstash.yml文件,里面定义好你的playbook

[root@master elk]# cat logstash.yml 
- hosts: logstash_servers
  remote_user: root
  roles:
    - logstash

5. 运行playbook安装logstash

[root@master elk]# ansible-playbook logstash.yml

 

八、es roles

1.把elasticsearch.tar.gz传到files目录下,es配置文件和启动脚本传到templates目录下

2.编写vars

[root@master es]# cat vars/main.yml 
es_package_file: elasticsearch.tar.gz
es_dir: /usr/local/elasticsearch
config_name: elasticsearch.yml

3.编写tasks

[root@master es]# cat tasks/main.yml 
- name: copy es software
  copy: src={{ es_package_file }} dest=/tmp/ owner=root group=root
- name: uncompression es software
  shell: tar zxf /tmp/{{ es_package_file }} -C /usr/local
- name: copy es start script
  template: src=start.sh dest={{ es_dir }}/ owner=es group=es mode=0755
- name: copy es config
  template: src=elasticsearch.yml dest={{ es_dir }}/config/ owner=es group=es
- name: update es config
  shell: sed -i "{{ item }}" {{ es_dir }}/config/{{ config_name }}
  with_items:
    - s/#node.name/node.name: {{ node_name }}/
    - s/#node.master/node.master: {{ node_master }}/
    - s/#node.data/node.data: {{ node_data }}/
    - s/#network.host/network.host: {{ network_host }}/
- name: create es user
  user: name=es
- name: change owner and group
  file: path={{ es_dir }} owner=es group=es #recurse=yes
- name: mkdir directory for es data
  file: dest=/data/elasticsearch mode=0755 state=directory owner=es group=es
  notify: mkdir Subdirectory
- name: start es service
  shell: su es -c cd {{ es_dir }} && ./start.sh
- name: delete es compression files
  shell: rm -f /tmp/{{ es_package_file }}

4.编写handlers

[root@master es]# cat handlers/main.yml 
- name: mkdir Subdirectory
  file: dest=/data/elasticsearch/{{ item }} mode=0755 state=directory owner=es group=es
  with_items:
    - logs
    - data

5.使用角色

在roles同级目录,创建一个es.yml文件,里面定义好你的playbook

[root@master elk]# cat es.yml 
- hosts: es_servers
  remote_user: root
  roles:
    - es

5. 运行playbook安装es

[root@master elk]# ansible-playbook es.yml

 

九、es-head roles

1.把node-v4.4.7-linux-x64.tar.gz和elasticsearch-head.tar.gz传到files目录下,把_site/app.js传到templates目录下

2.编写vars

[root@master es_head]# cat vars/main.yml 
node_package_file: node-v4.4.7-linux-x64.tar.gz
node_dir: /usr/local/node-v4.4.7-linux-x64
head_package_file: elasticsearch-head.tar.gz
head_dir: /usr/local/elasticsearch/elasticsearch-head

3.编写tasks

[root@master es_head]# cat tasks/main.yml 
- name: copy node software
  copy: src={{ node_package_file }} dest=/tmp/ owner=root group=root
- name: uncompression node software
  shell: tar zxf /tmp/{{ node_package_file }} -C /usr/local/
- name: copy node to env
  shell: rsync -a {{ node_dir  }}/bin/* /usr/bin/
- name: copy head plugin software
  copy: src={{ head_package_file }} dest=/tmp/ owner=root group=root
- name: uncopmression head plugin software
  shell: tar zxf /tmp/{{ head_package_file }} -C /usr/local/elasticsearch/
- name: copy head config
  template: src=app.js dest={{ head_dir }}/_site/ owner=es group=es
- name: change owner and group
  file: path={{ head_dir }} owner=es group=es #recurse=yes
- name: start es service
  shell: su es -c cd {{ head_dir }} && npm run start > {{ head_dir }}/run.log &
- name: delete node compression files
  shell: rm -f /tmp/{{ node_package_file }}
- name: delete head compression files
  shell: rm -f /tmp/{{ head_package_file }}

4.使用角色

在roles同级目录,创建一个es_head.yml文件,里面定义好你的playbook

[root@master elk]# cat es_head.yml 
- hosts: es_plugin_head
  remote_user: root
  roles:
    - es_head

5. 运行playbook安装es-head

ansible-playbook es_head.yml

 

十、kibana roles

1.把kibana.tar.gz传到files目录下,kibana配置文件和启动脚本传到templates目录下

2.编写vars

[root@master kibana]# cat vars/main.yml 
kibana_package_file: kibana.tar.gz
kibana_dir: /usr/local/kibana

3.编写tasks

[root@master kibana]# cat tasks/main.yml 
- name: copy kibana software
  copy: src={{ kibana_package_file }} dest=/tmp/ owner=root group=root
- name: uncompression kibana software
  shell: tar zxf /tmp/{{ kibana_package_file }} -C /usr/local
- name: copy kibana start script
  template: src=start.sh dest={{ kibana_dir }}/ owner=es group=es mode=0755
- name: copy kibana config
  template: src=kibana.yml dest={{ kibana_dir }}/config/ owner=es group=es
- name: change owner and group
  file: path={{ kibana_dir }} owner=es group=es
- name: start kibana service
  shell: su - es -c "{{ kibana_dir }}/start.sh"
- name: delete kibana compression files
  shell: rm -f /tmp/{{ kibana_package_file }}

4.使用角色

在roles同级目录,创建一个kibana.yml文件,里面定义好你的playbook

[root@master elk]# cat kibana.yml 
- hosts: kibana_servers
  remote_user: root
  roles:
    - kibana

5. 运行playbook安装kibana

ansible-playbook kibana.yml

 

十一、nginx roles(211本身就已经安装nginx,如果目标机器没安装,请参考我之前的文章)

1.把htpasswd生成密码的对应文件传到files目录下,nginx配置文件传到templates目录下

2.编写vars

[root@master nginx]# cat vars/main.yml 
nginx_basedir: /usr/local/nginx

3.编写tasks

[root@master nginx]# cat tasks/main.yml 
- name: copy conf file
  template: src={{ item.src }} dest={{ nginx_basedir }}/{{ item.dest }} backup=yes owner=root group=root mode=0644
  with_items:
    - { src: es.conf, dest: conf/vhosts/ }
    - { src: kibana.conf, dest: conf/vhosts/ }
- name: copy passwd file
  copy: src={{ item }} dest={{ nginx_basedir }}/conf/htpasswd/ owner=root group=root mode=0644
  with_items:
    - es
    - kibana
  notify: restart nginx

4.编写handlers

[root@master nginx]# cat handlers/main.yml 
- name: restart nginx
  shell: /usr/local/nginx/sbin/nginx -s reload

5.使用角色

在roles同级目录,创建一个nginx.yml文件,里面定义好你的playbook

[root@master elk]# cat nginx.yml 
---
- hosts: nginx_servers
  user: root
  roles:
    - nginx

5. 运行playbook加载nginx

ansible-playbook nginx.yml

ansible playbook部署ELK集群系统

标签:tor   htpasswd   密码   nginx配置   /etc/   脚本   pos   root   根据   

原文地址:http://www.cnblogs.com/654wangzai321/p/8046742.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!