码迷,mamicode.com
首页 > 其他好文 > 详细

elasticsearch的安装部署

时间:2019-12-29 16:57:43      阅读:91      评论:0      收藏:0      [点我收藏+]

标签:lease   modprobe   查询   nes   struct   effective   etc   ase   eps   

环境部署

1.服务器准备:centos7.4系统,Jdk1.8 cat /etc/redhat-releaseEs6.x

如果是自带的openjdk需要先卸载:

rpm -qa|grep java

rpm -e –nodeps *

技术图片

2.ElasticSearch安装:我采用的版本为6.6.0 https://elasticsearch.cn/download/

1)地址:https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.tar.gz 

(2)解压:tar -zxvf elasticsearch/elasticsearch-6.6.0.tar.gz

 (3)修改系统配置:

  • 设置内核参数 :vim /etc/sysctl.conf

     添加如下内容:
                fs.file-max=65536
                vm.max_map_count=262144
            sysctl -p 刷新下配置,

    sysctl -a查看是否生效  ,

如果不成功的(启动es还是失败,不是所有人都碰得到,好像是在7.6碰到了):

    rm -f /sbin/modprobe ?

    ln -s /bin/true /sbin/modprobe
    rm -f /sbin/sysctl 

    ?ln -s /bin/true /sbin/sysctl

  • 设置资源参数
                 vi /etc/security/limits.conf
                 # 添加一下内容:
                 * soft nofile 65536
                 * hard nofile 131072
                 * soft nproc  2048
                * hard nproc  4096

  • 修改进程数
                 vi /etc/security/limits.d/20-nproc.conf
                    *          soft    nproc     4096
    配置完成后 要关掉链接窗口,重新打开一个
    不能用root用户启动

  • 修改es配置文件: vim elasticsearch安装路径/config/elasticsearch.yml
    # 配置es的集群名称, es会自动发现在同一网段下的es,如果在同一网段下有多个集群,就可以用这个属性来区分不同的集群?
    cluster.name: bi-cluster
    # 节点名称
    node.name: node-master
    # 存储数据的目录
    path.data: /home/elasticsearch/data
    # 存储日志的目录
    path.logs: /home/elasticsearch/logs
    # 设置绑定的ip地址还有其它节点和该节点交互的ip地址
    network.host: 0.0.0.0
    # 指定http端口,你使用head?kopf等相关插件使用的端口
    http.port: 9200
    # 设置节点间交互的tcp端口,默认是9300
    transport.tcp.port: 9300
    #设置集群中master节点的初始列表,可以通过这些节点来自动发现新加入集群的节点?
    discovery.zen.ping.unicast.hosts: ["10.108.4.203:9300", "10.108.4.204:9300", "10.108.4.205:9300"]
    #如果没有这种设置,遭受网络故障的集群就有可能将集群分成两个独立的集群 - 分裂的大脑 - 这将导致数据丢失
    discovery.zen.minimum_master_nodes: 3
    http.cors.enabled: true        #es5.x版本以上需要,head访问
    http.cors.allow-origin: "*"
    bootstrap.memory_lock: false    #某些系统需要 是因为centos6.x操作系统不支持SecComp,而elasticsearch 5.5.2默认 bootstrap.system_call_filter为true进行检测,所以导致检测失败,失败后直接导致ES不能启动。
    我的配置如下:

    # ======================== Elasticsearch Configuration ====================
    =====
    #
    # NOTE: Elasticsearch comes with reasonable defaults for most settings.
    #       Before you set out to tweak and tune the configuration, make sure y
    ou
    #       understand what are you trying to accomplish and the consequences.
    #
    # The primary way of configuring a node is via this file. This template lis
    ts
    # the most important settings you may want to configure for a production cl
    uster.
    #
    # Please consult the documentation for further information on configuration
     options:
    # https://www.elastic.co/guide/en/elasticsearch/reference/index.html
    #
    # ---------------------------------- Cluster ------------------------------
    -----
    #
    # Use a descriptive name for your cluster:
    #
    #cluster.name: my-application
    #
    # ------------------------------------ Node -------------------------------
    -----
    #
    # Use a descriptive name for the node:
    #
    #node.name: node-1
    #
    # Add custom attributes to the node:
    #
    #node.attr.rack: r1
    #
    # ----------------------------------- Paths -------------------------------
    -----
    #
    # Path to directory where to store the data (separate multiple locations by
     comma):
    #
    #path.data: /path/to/data
    #
    # Path to log files:
    #
    #path.logs: /path/to/logs
    #
    # ----------------------------------- Memory ------------------------------
    -----
    #
    # Lock the memory on startup:
    #
    #bootstrap.memory_lock: true
    #
    # Make sure that the heap size is set to about half the memory available
    # on the system and that the owner of the process is allowed to use this
    # limit.
    #
    # Elasticsearch performs poorly when the system is swapping the memory.
    #
    # ---------------------------------- Network ------------------------------
    -----
    #
    # Set the bind address to a specific IP (IPv4 or IPv6):
    #
    #network.host: 192.168.0.1
    #
    # Set a custom port for HTTP:
    #
    #http.port: 9200
    #
    # For more information, consult the network module documentation.
    #
    # --------------------------------- Discovery -----------------------------
    -----
    #
    # Pass an initial list of hosts to perform discovery when new node is start
    ed:
    # The default list of hosts is ["127.0.0.1", "[::1]"]
    #
    #discovery.zen.ping.unicast.hosts: ["host1", "host2"]
    #
    # Prevent the "split brain" by configuring the majority of nodes (total num
    ber of master-eligible nodes / 2 + 1):
    #
    #discovery.zen.minimum_master_nodes: 
    #
    # For more information, consult the zen discovery module documentation.
    #
    # ---------------------------------- Gateway ------------------------------
    -----
    #
    # Block initial recovery after a full cluster restart until N nodes are sta
    rted:
    #
    #gateway.recover_after_nodes: 3
    #
    # For more information, consult the gateway module documentation.
    #
    # ---------------------------------- Various ------------------------------
    -----
    #
    # Require explicit names when deleting indices:
    #
    #action.destructive_requires_name: true
    cluster.name: my-es
    node.name: node-128
    network.host: 0.0.0.0
    http.port: 19200
    transport.tcp.port: 19300
    #discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
    #discovery.zen.minimum_master_nodes: 3
    http.cors.enabled: true        
    http.cors.allow-origin: "*"
    bootstrap.memory_lock: false    
    bootstrap.system_call_filter: false
    

      

  • 启动es:./bin/elasticsearch 后台启动加 –d参数

  • 打开浏览器访问 http://你的ip:19200/查看是否能够正常访问,看到以下界面表示启动ok

    技术图片

     

     

      

如果在外网访问失败的话,请按照以下步骤执行:

  1. 检查你的elasticsearch.yml配置文件的以下三处是否配置正确
    network.host: 0.0.0.0
    bootstrap.memory_lock: false    
    bootstrap.system_call_filter: false
  2. 检查防火墙是否关闭(需要关闭防火墙外网才可以访问)

    开启、重启、关闭、firewalld.service服务

    # 开启
    service firewalld start
    # 重启
    service firewalld restart
    # 关闭
    service firewalld stop
    查看防火墙规则:

    firewall-cmd --list-all
    查询、开放、关闭端口:

    # 查询端口是否开放
    firewall-cmd --query-port=8080/tcp
    # 开放80端口
    firewall-cmd --permanent --add-port=80/tcp
    # 移除端口
    firewall-cmd --permanent --remove-port=8080/tcp
    #重启防火墙(修改配置后要重启防火墙)
    firewall-cmd --reload

    执行以上步骤之后就可以在外网访问了。以下是我遇到的启动错误的问题以及解决办法:
  3. (1)max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

     每个进程最大同时打开文件数太小,可通过下面2个命令查看当前数量
  4. ulimit -Hn
    ulimit -Sn

      修改/etc/security/limits.conf文件,增加配置,用户退出后重新登录生效

    *               soft    nofile          65536
    *               hard    nofile          65536

    技术图片

     

    (2)max number of threads [3818] for user [es] is too low, increase to at least [4096]

      问题同上,最大线程个数太低。修改配置文件/etc/security/limits.conf(和问题1是一个文件),增加配置

    *               soft    nproc           4096
    *               hard    nproc           4096

      可通过命令查看

    ulimit -Hu
    ulimit -Su

    技术图片

    修改后的文件:

    技术图片

     如果更改后发现ulimit -Su 的结果没有变化的话,此时需要在这个文件中添加如下信息:(admin是我自己的用户名称,可以根据自己的实际用户更改)

    admin - nproc 4096

    技术图片

     

     

     

  5. max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

      修改/etc/sysctl.conf文件,增加配置vm.max_map_count=262144

    vi /etc/sysctl.conf
    sysctl -p

      执行命令sysctl -p生效

    技术图片

     4、Exception in thread "main" java.nio.file.AccessDeniedException: /usr/local/elasticsearch/elasticsearch-6.2.2-1/config/jvm.options

      elasticsearch用户没有该文件夹的权限,执行命令

    chown -R es:es /usr/local/elasticsearch/

安装kibana:

1、将下载的kibana安装包解压: tar -zxvf kibana-6.6.0-linux-x86_64.tar.gz

2、更改kibana的配置

# Kibana is served by a back end server. This setting specifies the port to
 use.
server.port: 15601

# Specifies the address to which the Kibana server will bind. IP addresses 
and host names are both valid values.
# The default is ‘localhost‘, which usually means remote machines will not 
be able to connect.
# To allow connections from remote users, set this parameter to a non-loopb
ack address.
#server.host: "localhost"
server.host: "0.0.0.0"


# Enables you to specify a path to mount Kibana at if you are running behin
d a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remo
ve the basePath
# from requests it receives, and to prevent a deprecation warning at startu
p.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse prox
y.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server‘s name.  This is used for display purposes.
#server.name: "your-hostname"

# The URLs of the Elasticsearch instances to use for all your queries.
#elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.hosts: ["http://localhost:19200"]


# When this setting‘s value is true Kibana uses the hostname specified in t
he server.host
# setting. When the value of this setting is false, Kibana uses the hostnam
e of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizat
ions and
# dashboards. Kibana creates a new index if the index doesn‘t already exist
.
#kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these setti
ngs provide
# the username and password that the Kibana server uses to perform maintena
nce on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elast
icsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files
, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to
 the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificat
e and key files.
# These files validate that your Elasticsearch backend uses the same key fi
les.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for t
he certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting‘s valu
e to ‘none‘.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defau
lts to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticse
arch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no*
 client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom header
s cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhi
telist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards.
 Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Time in milliseconds to wait for Elasticsearch at Kibana startup before r
etrying.
#elasticsearch.startupTimeout: 5000

[admin@localhost kibana-6.6.0-linux-x86_64]$ cat config/kibana.yml         
# Kibana is served by a back end server. This setting specifies the port to
 use.                                                                      
server.port: 15601                                                         
                                                                           
# Specifies the address to which the Kibana server will bind. IP addresses 
and host names are both valid values.                                      
# The default is ‘localhost‘, which usually means remote machines will not 
be able to connect.                                                        
# To allow connections from remote users, set this parameter to a non-loopb
ack address.                                                               
#server.host: "localhost"                                                  
server.host: "0.0.0.0"                                                     
                                                                           
                                                                           
# Enables you to specify a path to mount Kibana at if you are running behin
d a proxy.                                                                 
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remo
ve the basePath                                                            
# from requests it receives, and to prevent a deprecation warning at startu
p.                                                                         
# This setting cannot end in a slash.                                      
#server.basePath: ""                                                       
                                                                           
# Specifies whether Kibana should rewrite requests that are prefixed with  
# `server.basePath` or require that they are rewritten by your reverse prox
y.                                                                         
# This setting was effectively always `false` before Kibana 6.3 and will   
# default to `true` starting in Kibana 7.0.                                
#server.rewriteBasePath: false                                             

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server‘s name.  This is used for display purposes.
#server.name: "your-hostname"

# The URLs of the Elasticsearch instances to use for all your queries.
#elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.hosts: ["http://localhost:19200"]


# When this setting‘s value is true Kibana uses the hostname specified in t
he server.host
# setting. When the value of this setting is false, Kibana uses the hostnam
e of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizat
ions and
# dashboards. Kibana creates a new index if the index doesn‘t already exist
.
#kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these setti
ngs provide
# the username and password that the Kibana server uses to perform maintena
nce on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elast
icsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files
, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to
 the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificat
e and key files.
# These files validate that your Elasticsearch backend uses the same key fi
les.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for t
he certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting‘s valu
e to ‘none‘.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defau
lts to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticse
arch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no*
 client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom header
s cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhi
telist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards.
 Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Time in milliseconds to wait for Elasticsearch at Kibana startup before r
etrying.
#elasticsearch.startupTimeout: 5000

# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false

# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid

# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output othe
r than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system
 usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number
 formats.
#i18n.locale: "en"

3、启动kibana:  ./bin/kibana

技术图片

 

 4、外网通过ip地址访问:

技术图片

 

 

注意两个点:
(1)修改启动的es链接地址
(2)修改地址绑定不然只有本地才能访问

(3)kibana的版本和elasticsearch的版本必须一致,比如都是6.6

elasticsearch的安装部署

标签:lease   modprobe   查询   nes   struct   effective   etc   ase   eps   

原文地址:https://www.cnblogs.com/yatou-blog/p/12113909.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!