标签:system 合并 多个 ble user smtp 解析 eve lock
prometheus 通过alertmanager进行告警
实现监控告警的步骤:
rule_files
监控服务端主机下载安装alertmanager
alertmanager可以和prometheus主机安装在一起,也可以独立部署在一台主机。这里部署在一台主机
[root@localhost ~]# tar zxf alertmanager-0.19.0.linux-amd64.tar.gz
[root@localhost ~]# mv alertmanager-0.19.0.linux-amd64 /usr/local/alertmanager
alertmanager主配置文件: alertmanager.yml内容标注
global: # 全局配置
resolve_timeout: 5m #解析超时时间
route: # 配置告警发送,接受规则
group_by: ['alertname'] # 根据标签分组
group_wait: 10s # 发送告警等待时间,为了合并相同告警一起发送
group_interval: 10s # 发送告警间隔时间
repeat_interval: 10m # 重复告警时间,控制发送告警频率,根据情况设置
receiver: 'web.hook' # 定义接收者类型:mail,wechat等
receivers: # 定义告警发给谁
- name: 'web.hook'
webhook_configs:
- url: 'http://127.0.0.1:5001/'
inhibit_rules: # 告警抑制规则
- source_match:
severity: 'critical'
target_match:
severity: 'warning'
equal: ['alertname', 'dev', 'instance']
global:
resolve_timeout: 5m
smtp_smarthost: 'smtp.163.com:25' # 定义163邮箱服务器端
smtp_from: 'gudaoyufu@163.com' #来自哪个邮箱发的
smtp_auth_username: 'gudaoyufu@163.com' 邮箱验证
smtp_auth_password: 'XXXXXXXX' # 邮箱授权码,不是登录密码
smtp_require_tls: false # 是否启用tls
route:
group_by: ['alertname']
group_wait: 10s
group_interval: 10s
repeat_interval: 3m # 发送告警后间隔多久再次发送,减少发送邮件频率
receiver: 'mail' #发送的告警媒体
receivers:
- name: 'mail' # 接收者配置,这里要与接收媒体一致
email_configs:
- to: 'gudaoyufu@163.com' #发送给谁的邮箱,多个人多行列出
#inhibit_rules:
# - source_match:
# severity: 'critical'
# target_match:
# severity: 'warning'
# equal: ['alertname', 'dev', 'instance']
[root@localhost alertmanager]# ./amtool check-config alertmanager.yml
Checking 'alertmanager.yml' SUCCESS
Found:
- global config
- route
- 0 inhibit rules
- 1 receivers
- 0 templates
[root@localhost alertmanager]# ./alertmanager --config.file=./alertmanager.yml &
上面的配置只是配置alertmanagerr服务,接下来配置prometheus与alertmanager通信
https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/
[root@localhost ~]# cd /usr/local/prome/
[root@localhost prome]# vim prometheus.yml
#启用如需配置段
alerting:
alertmanagers:
- static_configs:
- targets:
- 127.0.0.1:9093
# 配置告警规则
rule_files:
- "rules.yml" # 默认路径在配置文件同级目录下
vim rules.yml
groups: # 报警组
- name: Node
rules:
# Alert for any instance that is unreachable for >5 minutes.
- alert: InstanceDown
expr: up == 0 # 监控状态的值为 0时,说明服务异常,1为正常
for: 5m # 保持时间,上面的状态持续时间内都为 0 ,则触发告警
labels:
severity: error
annotations:
summary: "Instance {{ $labels.instance }} 停止工作"
description: "{{ $labels.instance }} of job {{ $labels.job }} 已经停止1分钟以上."
[root@localhost prome]# ./promtool check config prometheus.yml
Checking prometheus.yml
SUCCESS: 1 rule files found
Checking rules.yml
SUCCESS: 1 rules found
systemctl start prometheus
重启prometheus后在prometheus服务端页面中的alters中可以看到rules
#停掉cadvisor
[root@localhost ~]# docker stop e9e9499bcf2b
e9e9499bcf2b
一会后http://192.168.235.130:9090/alerts
中的状态为Firing
,此时邮件已经发出去了,登录邮箱已经看到告警邮件了
Firing : 已经触发阈值并且满足告警持续时间,警报发送给了接收者
分组: 将类似性质的告警分类成单个通知
route: # 配置告警发送,接受规则
group_by: ['alertname'] # 根据标签分组
group_wait: 10s # 发送告警等待时间,为了合并相同告警一起发送
group_interval: 10s # 发送告警间隔时间
repeat_interval: 10m # 重复告警时间,控制发送告警频率,根据情况设置
inhibit_rules:
- source_match: # 高级别告警源
severity: 'critical'
target_match: # 低级别的告警被抑制不会发送
severity: 'warning'
equal: ['alertname', 'dev', 'instance'] #抑制匹配
配置静默通过alertmanager服务的9093端口服务,界面创建静默规则
http://192.168.235.130:9093/#/silences
标签:system 合并 多个 ble user smtp 解析 eve lock
原文地址:https://www.cnblogs.com/anay/p/11871018.html