node-exporter配置白名单

1.禁用默认的采集模块

[root@node-exporter41 node_exporter-1.9.0.linux-amd64]# ./node_exporter --collector.disable-defaults
time=2025-03-28T02:14:08.819Z level=INFO source=node_exporter.go:216 msg="Starting node_exporter" version="(version=1.9.0, branch=HEAD, revision=02afa5c53c36123611533f2defea6ccd4546a9bb)"
time=2025-03-28T02:14:08.819Z level=INFO source=node_exporter.go:217 msg="Build context" build_context="(go=go1.23.6, platform=linux/amd64, user=root@745e986f4d3a, date=20250217-07:22:48, tags=unknown)"
time=2025-03-28T02:14:08.819Z level=WARN source=node_exporter.go:219 msg="Node Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
time=2025-03-28T02:14:08.820Z level=INFO source=node_exporter.go:135 msg="Enabled collectors"
time=2025-03-28T02:14:08.820Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100
time=2025-03-28T02:14:08.820Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9100

2.配置白名单

[root@node-exporter41 node_exporter-1.9.0.linux-amd64]# ./node_exporter --collector.disable-defaults --collector.cpu --collector.os --collector.processes
time=2025-03-28T02:20:26.494Z level=INFO source=node_exporter.go:216 msg="Starting node_exporter" version="(version=1.9.0, branch=HEAD, revision=02afa5c53c36123611533f2defea6ccd4546a9bb)"
time=2025-03-28T02:20:26.494Z level=INFO source=node_exporter.go:217 msg="Build context" build_context="(go=go1.23.6, platform=linux/amd64, user=root@745e986f4d3a, date=20250217-07:22:48, tags=unknown)"
time=2025-03-28T02:20:26.494Z level=WARN source=node_exporter.go:219 msg="Node Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
time=2025-03-28T02:20:26.495Z level=INFO source=node_exporter.go:135 msg="Enabled collectors"
time=2025-03-28T02:20:26.495Z level=INFO source=node_exporter.go:141 msg=cpu
time=2025-03-28T02:20:26.495Z level=INFO source=node_exporter.go:141 msg=os
time=2025-03-28T02:20:26.495Z level=INFO source=node_exporter.go:141 msg=processes
time=2025-03-28T02:20:26.495Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100
time=2025-03-28T02:20:26.495Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9100

3.配置黑名单

[root@node-exporter41 node_exporter-1.9.0.linux-amd64]# ./node_exporter  --no-collector.cpu --no-collector.os
time=2025-03-28T02:18:32.947Z level=INFO source=node_exporter.go:216 msg="Starting node_exporter" version="(version=1.9.0, branch=HEAD, revision=02afa5c53c36123611533f2defea6ccd4546a9bb)"
time=2025-03-28T02:18:32.948Z level=INFO source=node_exporter.go:217 msg="Build context" build_context="(go=go1.23.6, platform=linux/amd64, user=root@745e986f4d3a, date=20250217-07:22:48, tags=unknown)"
time=2025-03-28T02:18:32.948Z level=WARN source=node_exporter.go:219 msg="Node Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
time=2025-03-28T02:18:32.948Z level=INFO source=filesystem_common.go:266 msg="Parsed flag --collector.filesystem.mount-points-exclude" collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/)
time=2025-03-28T02:18:32.948Z level=INFO source=filesystem_common.go:295 msg="Parsed flag --collector.filesystem.fs-types-exclude" collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$
time=2025-03-28T02:18:32.948Z level=INFO source=diskstats_common.go:110 msg="Parsed flag --collector.diskstats.device-exclude" collector=diskstats flag=^(z?ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:135 msg="Enabled collectors"
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=arp
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=bcache
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=bonding
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=btrfs
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=conntrack
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=cpufreq
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=diskstats
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=dmi
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=edac
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=entropy
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=fibrechannel
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=filefd
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=filesystem
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=hwmon
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=infiniband
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=ipvs
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=loadavg
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=mdadm
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=meminfo
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=netclass
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=netdev
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=netstat
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=nfs
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=nfsd
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=nvme
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=powersupplyclass
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=pressure
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=rapl
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=schedstat
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=selinux
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=sockstat
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=softnet
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=stat
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=tapestats
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=textfile
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=thermal_zone
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=time
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=timex
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=udp_queues
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=uname
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=vmstat
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=watchdog
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=xfs
time=2025-03-28T02:18:32.949Z level=INFO source=node_exporter.go:141 msg=zfs
time=2025-03-28T02:18:32.949Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100
time=2025-03-28T02:18:32.949Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9100

4.官方默认开启的模块

https://github.com/prometheus/node_exporter?tab=readme-ov-file#enabled-by-default

Prometheus server服务端配置白名单

1.修改Prometheus的配置文件

[root@prometheus-server31 ~]# tail -12 /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml
  - job_name: violet-linux-node-exporter
    # 添加参数
    params:
      # 仅采集cpu和内存相关的模块数据
      collect[]:
        - cpu
        - meminfo
    static_configs:
      - targets: 
        - 10.0.0.41:9100
        - 10.0.0.42:9100
        - 10.0.0.43:9100
[root@prometheus-server31 ~]#

2.热加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

3.Prometheus的WebUI查询数据验证

node_os_info  ---> OS模块 
node_cpu_seconds_total  ---> CPU模块 
node_memory_MemTotal_bytes ---> memory模块

Prometheus server服务端配置白名单

1.修改Prometheus的配置文件

[root@prometheus-server31 ~]# tail -16 /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml
  - job_name: violet-linux-node-exporter
    # 添加参数
    params:
      # 添加白名单，仅采集cpu和内存相关的模块数据
      # collect[]:
      #   - cpu
      #   - meminfo
      # 添加黑名单，排除这两个模块的数据采集
      exclude[]:
        - cpu
        - meminfo
    static_configs:
      - targets: 
        - 10.0.0.41:9100
        - 10.0.0.42:9100
        - 10.0.0.43:9100
[root@prometheus-server31 ~]# 

	
[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]# 


	3.Prometheus的WebUI查询数据验证 
node_os_info  ---> OS模块 
node_cpu_seconds_total  ---> CPU模块 
node_memory_MemTotal_bytes ---> memory模块

2.热加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

3.Prometheus的WebUI查询数据验证

node_os_info  ---> OS模块 
node_cpu_seconds_total  ---> CPU模块 
node_memory_MemTotal_bytes ---> memory模块

Prometheus的标签管理

1.服务端支持自定义标签

[root@prometheus-server31 ~]# tail -11 /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml
  - job_name: violet-linux-node-exporter-lables
    static_configs:
      - targets: 
        - 10.0.0.41:9100
        - 10.0.0.42:9100
        - 10.0.0.43:9100
        # 给target添加标签
        labels:
          school: violet
          class: linux
          apps: bigdata
[root@prometheus-server31 ~]#

2.热加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

3.验证测试

node_os_info  ---> OS模块 
node_cpu_seconds_total  ---> CPU模块 
node_memory_MemTotal_bytes ---> memory模块

服务发现

静态发现
- 动态发现

relabel_configs为target重新打标签

1.添加配置

[root@prometheus-server31 ~]# tail -30 /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml 

  - job_name: violet-linux-node-exporter-lables-xixi
    # 重新打标签的相关配置
    relabel_configs:
      # 将源标签的值取出来
    - source_labels: 
      - class
      - school
      - apps
      # 将source_labels的值使用separator分隔符进行拼接，如果不定义默认使用";"来拼接
      separator: "---"
      # 从source_labels提取的值进行正则表达式匹配，如果不定义，默认使用"(.*)"匹配
      regex: (.*)
      # regex: "(.{5}96)---(.{6}edu)---(big.{4})"
      # 定义要处理的动作，若不指定，默认值为: "replace"
      #   https://prometheus.io/docs/prometheus/2.53/configuration/configuration/#relabel_action
      action: uppercase
      # 引用regex数据作为值传递给target_label，若不定义，则默认为"$1"
      replacement: $1
      # 将转换后的数据存储的标签
      target_label: laonanhai
    static_configs:
      - targets: 
        - 10.0.0.41:9100
        - 10.0.0.42:9100
        # 给target添加标签
        labels:
          school: violet
          class: linux
          apps: bigdata
[root@prometheus-server31 ~]#

2.测试访问

node_cpu_guest_seconds_total

部署blackbox-exporter黑盒监控

1.blackbox-exporter概述

一般用于监控网站是否监控，端口是否存活，证书有效期等。


blackbox exporter支持基于HTTP, HTTPS, DNS, TCP, ICMP, gRPC协议来对目标节点进行监控。

比如基于http协议我们可以探测一个网站的返回状态码为200判读服务是否正常。

比如基于TCP协议我们可以探测一个主机端口是否监听。

比如基于ICMP协议来ping一个主机的连通性。

比如基于gRPC协议来调用接口并验证服务是否正常工作。

比如基于DNS协议可以来检测域名解析。

2.下载blackbox-exporter

wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.26.0/blackbox_exporter-0.26.0.linux-amd64.tar.gz

3.解压软件包

[root@node-exporter41 ~]# tar xf blackbox_exporter-0.26.0.linux-amd64.tar.gz  -C /usr/local/
[root@node-exporter41 ~]# 
[root@node-exporter41 ~]# cd /usr/local/blackbox_exporter-0.26.0.linux-amd64/
[root@node-exporter41 blackbox_exporter-0.26.0.linux-amd64]# ll
total 23232
drwxr-xr-x  2 1001 1002     4096 Feb 26 20:31 ./
drwxr-xr-x 11 root root     4096 Mar 28 11:55 ../
-rwxr-xr-x  1 1001 1002 23758662 Feb 26 20:30 blackbox_exporter*
-rw-r--r--  1 1001 1002     1209 Feb 26 20:31 blackbox.yml
-rw-r--r--  1 1001 1002    11357 Feb 26 20:31 LICENSE
-rw-r--r--  1 1001 1002       94 Feb 26 20:31 NOTICE
[root@node-exporter41 blackbox_exporter-0.26.0.linux-amd64]#

4.启动blackbox服务

[root@node-exporter41 blackbox_exporter-0.26.0.linux-amd64]# ./blackbox_exporter

5.访问blackbox的WebUI

http://10.0.0.41:9115/

6.访问测试

http://10.0.0.41:9115/probe?target=www.violet.com&module=http_2xx
http://10.0.0.41:9115/probe?target=prometheus.io&module=http_2xx

Prometheus server整合blackbox实现网站监控

1.修改Prometheus的配置文件

[root@prometheus-server31 ~]# tail -35 /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml 
    # 指定作业的名称，生成环境中，通常是指一类业务的分组配置。
  - job_name: 'violet-blackbox-exporter-http'
    # 修改访问路径，若不修改，默认值为"/metrics"
    metrics_path: /probe
    # 配置URL的相关参数
    params:
      # 此处表示使用的是blackbox的http模块，从而判断相应的返回状态码是否为200
      module: [http_2xx]
      school: [violet]
      class: ["linux"]
    # 静态配置，需要手动指定监控目标
    static_configs:
        # 需要监控的目标
      - targets:
          # 支持https协议
        - https://www.violet.com/
          # 支持http协议
        - http://10.0.0.41
          # 支持http协议和自定义端口
        - http://10.0.0.31:9090
    # 对目标节点进行重新打标签配置
    relabel_configs:
        # 指定源标签，此处的"__address__"表示内置的标签，存储的是被监控目标的IP地址
      - source_labels: [__address__]
        # 指定目标标签，其实就是在"Endpoint"中加了一个target字段(用于指定监控目标)，
        target_label: __param_target
        # 指定需要执行的动作，默认值为"replace"，常用的动作有: replace, keep, and drop。
        # 但官方支持十几种动作： https://prometheus.io/docs/prometheus/2.45/configuration/configuration/
        # 将"__address__"传递给target字段。
        action: replace
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        # 指定要替换的值,此处我指定为blackbox exporter的主机地址
        replacement: 10.0.0.41:9115
[root@prometheus-server31 ~]#

2.热加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload

3.验证webUI

http://10.0.0.31:9090/targets?search=

4.导入grafana的模板ID

7587
13659

prometheus基于blackbox的ICMP监控目标主机是否存活

1 修改Prometheus配置文件

[root@prometheus-server31 ~]# vim /violet/softwares/prometheus-2.53.4.linux-amd64/prometheus.yml 
...
scrape_configs:
  ...
  - job_name: 'violet-blackbox-exporter-icmp'
    metrics_path: /probe
    params:
      # 如果不指定模块，则默认类型为"http_2xx"，不能乱写!乱写监控不到服务啦!
      module: [icmp]
    static_configs:
      - targets:
          - 10.0.0.41
          - 10.0.0.42
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        # 指定注意的是，如果instance不修改，则instance和"__address__"的值相同
        # target_label: ip
        target_label: instance
      - target_label: __address__
        replacement: 10.0.0.41:9115

2 检查配置文件是否正确

[root@prometheus-server31 ~]# cd /violet/softwares/prometheus-2.53.4.linux-amd64/
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# ll
total 261348
drwxr-xr-x 5 1001 fwupd-refresh      4096 Mar 28 14:35 ./
drwxr-xr-x 3 root root               4096 Mar 26 09:45 ../
drwxr-xr-x 2 1001 fwupd-refresh      4096 Mar 18 23:05 console_libraries/
drwxr-xr-x 2 1001 fwupd-refresh      4096 Mar 18 23:05 consoles/
drwxr-xr-x 4 root root               4096 Mar 26 14:49 data/
-rw-r--r-- 1 1001 fwupd-refresh     11357 Mar 18 23:05 LICENSE
-rw-r--r-- 1 1001 fwupd-refresh      3773 Mar 18 23:05 NOTICE
-rwxr-xr-x 1 1001 fwupd-refresh 137836884 Mar 18 22:52 prometheus*
-rw-r--r-- 1 1001 fwupd-refresh      4858 Mar 28 14:35 prometheus.yml
-rw-r--r-- 1 root root               1205 Mar 27 10:05 prometheus.yml2025-03-26
-rw-r--r-- 1 root root               2386 Mar 28 10:06 prometheus.yml2025-03-27
-rwxr-xr-x 1 1001 fwupd-refresh 129719117 Mar 18 22:52 promtool*
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# 
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# ./promtool check config prometheus.yml
Checking prometheus.yml
 SUCCESS: prometheus.yml is valid prometheus config file syntax

3 重新加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

4 访问prometheus的WebUI

http://10.0.0.31:9090/targets

5 访问blackbox的WebUI

http://10.0.0.41:9115/

6 grafana过滤jobs数据

基于"violet-blackbox-exporter-icmp"标签进行过滤

prometheus基于blackbox的TCP案例监控端口是否存活

1 修改Prometheus配置文件

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# vim prometheus.yml

...
scrape_configs:
  ...
  - job_name: 'violet-blackox-exporter-tcp'
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
      - targets:
          - 10.0.0.41:80
          - 10.0.0.42:22
          - 10.0.0.31:9090
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.0.0.41:9115

2 检查配置文件是否正确

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# ./promtool check config prometheus.yml
Checking prometheus.yml
 SUCCESS: prometheus.yml is valid prometheus config file syntax

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]#

3 重新加载配置文件

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

4 访问prometheus的WebUI

http://10.0.0.31:9090/targets

5 访问blackbox exporter的WebUI

http://10.0.0.41:9115/

6 使用grafana查看数据

基于"oldboyedu-blackbox-exporter-tcp"标签进行过滤。

Prometheus的服务发现(Service Discovery，简称”sd”)

静态配置
修改配置后，需要热加载服务或者重启Prometheus服务，才能让配置生效。
- 动态配置
  修改配置后，不需要热加载或者重启就能生效。

Prometheus基于文件的服务发现

1 架构说明

2 修改prometheus的配置文件

[root@prometheus-server31 ~]# vim /violet/softwares/prometheus/prometheus.yml
...
scrape_configs:
  ...
  - job_name: 'violet-file-service-discovery-json'
    # 基于文件的服务发现
    file_sd_configs:
        # 指定文件路径
      - files:
          - /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.json

  - job_name: 'violet-file-service-discovery-yaml'
    file_sd_configs:
      - files:
          - /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.yaml

3 重新加载配置

curl -X POST http://10.0.0.31:9090/-/reload

4 修改动态配置文件

cat > /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.json <<EOF
[
  {
    "targets": [ "10.0.0.41:9100"],
    "labels": {
      "school": "violet", "class": "linux"
    }
  }
]
EOF


cat > /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.yaml <<EOF
- targets:
    - '10.0.0.42:9100'
    - '10.0.0.43:9100'
  labels:
    "address": "shahe"
    "office": "www.violet.com" 
    "class": "linux"
EOF

5 访问webUI

http://10.0.0.31:9090/targets?search=

基于consul服务实现自动发现

1 架构说明

2 部署consul集群

官方文档:
	https://www.consul.io/
	https://developer.hashicorp.com/consul/install#linux

                2.1 下载consul
wget https://releases.hashicorp.com/consul/1.20.5/consul_1.20.5_linux_amd64.zip  

svip:
[root@node-exporter41 ~]# wget http://192.168.16.253/Resources/Prometheus/softwares/Consul/consul_1.20.5_linux_amd64.zip


		2.2 解压consul
[root@node-exporter41 ~]# unzip consul_1.20.5_linux_amd64.zip -d /usr/local/bin/


		2.3 将consul环境拷贝到其他节点 
[root@node-exporter41 ~]# scp /usr/local/bin/consul 10.0.0.42:/usr/local/bin
[root@node-exporter41 ~]# scp /usr/local/bin/consul 10.0.0.43:/usr/local/bin


		2.4 运行consul 集群

			服务端43:
consul agent -server -bootstrap -bind=10.0.0.43 -data-dir=/violet/softwares/consul -client=10.0.0.43 -ui


			客户端42:
consul agent  -bind=10.0.0.42 -data-dir=/violet/softwares/consul -client=10.0.0.42 -ui -retry-join=10.0.0.43


			客户端41:
consul agent -server -bind=10.0.0.41 -data-dir=/violet/softwares/consul -client=10.0.0.41 -ui -retry-join=10.0.0.43

	
	
		2.5 查看各节点的监听端口
ss -ntl | grep 8500

		2.6 访问console服务的WebUI

http://10.0.0.41:8500/ui/dc1/nodes
http://10.0.0.42:8500/ui/dc1/nodes
http://10.0.0.43:8500/ui/dc1/nodes

Prometheus基于consul实现自动发现

1 修改prometheus的配置文件

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# vim prometheus.yml
...
  - job_name: "violet-consul-seriver-discovery"
    # 配置基于consul的服务发现
    consul_sd_configs:
        # 指定consul的服务器地址，若不指定，则默认值为"localhost:8500".
      - server: 10.0.0.43:8500
      - server: 10.0.0.42:8500
      - server: 10.0.0.41:8500
    relabel_configs:
        # 匹配consul的源标签字段，表示服务名称
      - source_labels: [__meta_consul_service]
        # 指定源标签的正则表达式，若不定义，默认值为"(.*)"
        regex: consul
        # 执行动作为删除，默认值为"replace",有效值有多种
        #   https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_action
        action: drop

2 检查配置文件是否正确

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# ./promtool  check config  prometheus.yml
Checking prometheus.yml
 SUCCESS: prometheus.yml is valid prometheus config file syntax

[root@prometheus-server31 prometheus-2.53.4.linux-amd64]#

3 重新加载配置

[root@prometheus-server31 ~]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 ~]#

4.被监控节点注册到console集群

                 4.1 注册节点 
[root@node-exporter43 ~]# curl -X PUT -d '{"id":"prometheus-node42","name":"oldboyedu-prometheus-node42","address":"10.0.0.42","port":9100,"tags":["node-exporter"],"checks": [{"http":"http://10.0.0.42:9100","interval":"5m"}]}' http://10.0.0.42:8500/v1/agent/service/register


注意，也可以使用POSTMAN直接进行如下操作
PUT http://10.0.0.43:8500/v1/agent/service/register
{"id":"prometheus-node43","name":"oldboyedu-prometheus-node43","address":"10.0.0.43","port":9100,"tags":["node-exporter"],"checks": [{"http":"http://10.0.0.43:9100","interval":"5m"}]}
 

		4.2 注销节点
[root@grafana71 ~]# curl -X PUT http://10.0.0.43:8500/v1/agent/service/deregister/prometheus-node42


 

TODO---> 目前有个坑
	你注册时找得哪个节点，那么注销时也要找这个节点注销，待解决...


注意，也可以使用POSTMAN直接进行如下操作，需要使用PUT方法。
PUT http://10.0.0.43:8500/v1/agent/service/deregister/prometheus-node43

Prometheus部署联邦模式

联邦模式图解

1 prometheus server32配置

                1.1 修改prometheus server配置文件
[root@prometheus-server32 prometheus-2.53.4.linux-amd64]# vim prometheus.yml 

  ...
  - job_name: 'violet-file-service-discovery-32'
    file_sd_configs:
      - files:
          - /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.yaml

		1.2 重载prometheus server
[root@prometheus-server32 prometheus-2.53.4.linux-amd64]# curl -X POST http://10.0.0.32:9090/-/reload
[root@prometheus-server32 prometheus-2.53.4.linux-amd64]# 


		1.3 编写yaml文件
[root@prometheus-server32 prometheus-2.53.4.linux-amd64]# cat > /violet/softwares/prometheus-2.53.4.linux-amd64/violet-file-sd.yaml <<EOF
- targets:
    - '10.0.0.41:9100'
  labels:
    "address": "shahe"
    "office": "www.violet.com" 
    "apps": "yaml"
EOF

		1.4 验证数据是否采集成功
http://10.0.0.32:9090/config
http://10.0.0.32:9090/targets?search=

2 prometheus server33 配置

                2.1 修改prometheus server的配置文件
[root@prometheus-server33 prometheus-2.53.4.linux-amd64]# vim prometheus.yml 
  ...
  - job_name: "violet-consul-seriver-discovery"
    consul_sd_configs:
      - server: 10.0.0.43:8500
      - server: 10.0.0.42:8500
      - server: 10.0.0.41:8500
    relabel_configs:
      - source_labels: [__meta_consul_service]
        regex: consul
        action: drop


		2.2 重载prometheus server
[root@prometheus-server33 prometheus-2.53.4.linux-amd64]# curl -X POST http://10.0.0.33:9090/-/reload
[root@prometheus-server33 prometheus-2.53.4.linux-amd64]# 

 

		2.3 注册节点 
[root@node-exporter43 ~]# curl -X PUT -d '{"id":"prometheus-node42","name":"violet-prometheus-node42","address":"10.0.0.42","port":9100,"tags":["node-exporter"],"checks": [{"http":"http://10.0.0.42:9100","interval":"5m"}]}' http://10.0.0.42:8500/v1/agent/service/register


[root@node-exporter43 ~]# curl -X PUT -d '{"id":"prometheus-node43","name":"violet-prometheus-node43","address":"10.0.0.43","port":9100,"tags":["node-exporter"],"checks": [{"http":"http://10.0.0.43:9100","interval":"5m"}]}' http://10.0.0.42:8500/v1/agent/service/register



		
		2.4 验证数据是否采集成功
http://10.0.0.33:9090/targets?search=
http://10.0.0.42:8500/ui/dc1/services

3 Prometheus server31配置

                 3.1 修改prometheus server的配置文件
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# vim prometheus.yml
...
  - job_name: "prometheus-federate-32"
    metrics_path: "/federate"
    # 用于解决标签的冲突问题，有效值为: true和false，默认值为false
    # 当设置为true时，将保留抓取的标签以忽略服务器自身的标签。说白了会覆盖原有标签。
    # 当设置为false时，则不会覆盖原有标签，而是在标点前加了一个"exported_"前缀。
    honor_labels: true
    params:
       "match[]":
       - '{job="promethues"}'
       - '{__name__=~"job:.*"}'
       - '{__name__=~"node.*"}'
    static_configs:
    - targets:
        - "10.0.0.32:9090"

  - job_name: "prometheus-federate-33"
    metrics_path: "/federate"
    honor_labels: true
    params:
       "match[]":
       - '{job="promethues"}'
       - '{__name__=~"job:.*"}'
       - '{__name__=~"node.*"}'
    static_configs:
    - targets:
        - "10.0.0.33:9090"


		3.2 重载prometheus server
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# curl -X POST http://10.0.0.31:9090/-/reload
[root@prometheus-server31 prometheus-2.53.4.linux-amd64]# 


		3.3 验证数据是否采集成功
node_cpu_seconds_total{job=~"oldboyedu-.*"}

发表回复取消回复

Related News

Prometheus监控es集群

Prometheus的快速安装和使用

Prometheus的Altermanager组件实现告警，配置自定义告警模板，以及结合钉钉插件如何实现告警

You may have missed

kafka单点部署-和kafka集群搭建

zookeeper单点部署以及集群搭建

Kibana如何部署

Prometheus监控es集群

归档

其他

node-exporter配置白名单

Prometheus server服务端配置白名单

Prometheus server服务端配置白名单

Prometheus的标签管理

服务发现

部署blackbox-exporter黑盒监控

Prometheus server整合blackbox实现网站监控

prometheus基于blackbox的ICMP监控目标主机是否存活

prometheus基于blackbox的TCP案例监控端口是否存活

Prometheus的服务发现(Service Discovery，简称”sd”)

Prometheus基于文件的服务发现

基于consul服务实现自动发现

Prometheus基于consul实现自动发现

Prometheus部署联邦模式

发表回复 取消回复

Related News

Prometheus监控es集群

Prometheus的快速安装和使用

Prometheus的Altermanager组件实现告警，配置自定义告警模板，以及结合钉钉插件如何实现告警

You may have missed

kafka单点部署-和kafka集群搭建

zookeeper单点部署以及集群搭建

Kibana如何部署

Prometheus监控es集群

发表回复取消回复