Prometheus架构从入门到实践(8) --alertman

1.RocketChat的webhook设置

      webhook设置参考URL：`https://github.com/pavel-kazhavets/AlertmanagerRocketChat`

1. Login as admin user and go to: Administration => Integrations => New Integration => Incoming WebHook

2. Set "Enabled" and "Script Enabled" to "True"

3. Set all channel, icons, etc. as you need

4. Paste contents of [AlertmanagerIntegrations.js](https://github.com/pavel-kazhavets/AlertmanagerRocketChar/blob/master/AlertmanagerIntegration.js) into Script field.

5. Create Integration. You;ll see some values apper. Copy WebHook URL and proceed to Alertmanager.

测试RocketChat webhook接口

  # curl -X POST -H 'Content-Type: application/json' --data '

{

  "text": "Example message",

  "attachments": [

    {

      "title": "Rocket.Chat",

      "title_link": "https://rocket.chat",

      "text": "Rocket.Chat, the best open source chat",

      "image_url": "https://rocket.cha t/images/mockup.png",

      "color": "#764FA5"

    }

  ],

  "status": "firing",

  "alerts": [

    {

      "labels": {

        "alertname": "Node Explorter Down",

        "instance": "192.168.8.87:9182",

        "job": "node.87",

        "severity": "disable"

      },

      "annotations": {

        "description": "192.168.8.87:9182 of job node.87is under Node Explorter Down.",

        "severity": "major",

        "summary": "node-exporter:9100 under high load."

      }

    }

  ]

}

' http://211.149.224.155:3000/hooks/ZanvEXd5t2Qaoycvm/Wwff7ndDeCq8DqFnaGhi4gdGWHpXJnaWyQoQzESypwwBvApJ

返回：{"success":true}

2.alertmanager配置变更

      参考URL：https://prometheus.io/docs/alerting/latest/configuration/#webhook_config

      # vi /root/alertmanager/alertmanager.yml

global:

  resolve_timeout: 5m

  http_config: 代理环境，所以需要该全局配置；也可以设置在具体receivers下面

    proxy_url: 'http://192.168.13.190:7777'

route:

  group_by: ['alertname']

  group_wait: 10s

  group_interval: 10s

  repeat_interval: 1m

  receiver: 'rocketchat'

receivers:

  - name: 'rocketchat'

    webhook_configs:

      - url: 'http://211.149.224.155:3000/hooks/ZanvEXd5t2Qaoycvm/Wwff7ndDeCq8DqFnaGhi4gdGWHpXJnaWyQoQzESypwwBvApJ'

        send_resolved: true

主要配置的作用：

global: 全局配置，包括报警解决后的超时时间、SMTP 相关配置、各种渠道通知的 API 地址等等。

route: 用来设置报警的分发策略，它是一个树状结构，按照深度优先从左向右的顺序进行匹配。

receivers: 配置告警消息接受者信息，例如常用的 email、wechat、slack、webhook 等消息通知方式。

检查alertmanager配置文件

      # /root/alertmanager/amtool check-config alertmanager.yml

Checking 'alertmanager.yml'  SUCCESS

Found:

- global config

- route

- 0 inhibit rules

- 1 receivers

- 0 templates

  SUCCESS

      # systemctl restart alertmanager

      # systemctl status alertmanager.service

3.告警测试

停止某个exporter

启动停止exporter