prometheus+timescaledb+pg_promet
2019-11-21 本文已影响0人
于飞_d529
版本
centos7
postgresqp-11
timescaledb-11
llvm5.0
clang5.0
环境准备
- 安装gcc-c++
yum install -y gcc gcc-c++
- 安装cmark
wget https://github.com/Kitware/CMake/releases/download/v3.16.0-rc2/cmake-3.16.0-rc2-Linux-x86_64.sh
mv cmake-3.16.0-rc2-Linux-x86_64.sh /usr/local/
#加入环境变量
export PATH=/usr/local/cmake-3.16.0-rc2-Linux-x86_64/bin:$PATH
prometheus安装
- 下载
https://prometheus.io/download/
- 解压安装
tar -zxvf prometheus-2.12.0.linux-amd64.tar.gz
- 启动
./prometheus --config.file="prometheus.yml" --web.listen-address="0.0.0.0:24003"
- web访问地址
http://192.168.20.31:24003
postgresql安装
yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
- 安装postgres11
yum install postgresql11
- 安装服务端
yum install postgresql11-server
- 设置开机启动和启动
/usr/pgsql-11/bin/postgresql-11-setup initdb
systemctl enable postgresql-11
systemctl start postgresql-11
- 写入环境变量
export PATH=$PATH:/usr/pgsql-11/bin
timescaledb安装
- 添加yum源
sudo cat > /etc/yum.repos.d/timescale_timescaledb.repo <<EOL
[timescale_timescaledb]
name=timescale_timescaledb
baseurl=https://packagecloud.io/timescale/timescaledb/el/7/\$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/timescale/timescaledb/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
EOL
- 更新yum源
sudo yum update -y
- 安装
sudo yum install -y timescaledb-postgresql-11
- 添加动态库参数
打开postgresql.conf 文件添加一行参数
shared_preload_libraries = ‘timescaledb’
- 重启pg
systemctl restart postgresql-11
- 创建timescaledb扩展
[root@MiWiFi-R3L-srv ~]# psql -h 127.0.0.1 -U postgres
postgres=# CREATE EXTENSION timescaledb;
llvm5.0+clang安装
由于编译pg_prometheus的需要，需要先手动编译 llvm5.0+clang，下载clang-tools-extra-5.0.0.src.tar.xz，compiler-rt-5.0.0.src.tar.xz，llvm-5.0.0.src.tar.xz
- 下载地址
http://releases.llvm.org/download.html#5.0.0 - 解压后将文件移动到制定目录下，并编译，大概需要1小时
export PATH=$PATH:/usr/local/cmake/bin
mv cfe-5.0.0.src clang
mv clang/ llvm-5.0.0.src/tools/
mv clang-tools-extra-5.0.0.src extra
mv extra/ llvm-5.0.0.src/tools/clang/
mv compiler-rt-5.0.0.src compiler-rt
mv compiler-rt-5.0.0.src/projects/
mkdir build-5.0.0
cd build-5.0.0
cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local/clang -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On ../llvm-5.0.0.src
make -j 4
make install
pg_prometheus安装
- 参考地址
https://github.com/timescale/pg_prometheus - 下载源码包并编译
make
make install
- 创建pg_prometheus扩展
在postgresql的postgresql.conf中添加并重启psql：
shared_preload_libraries = 'pg_prometheus'
- 授权role admin
GRANT ALL ON SCHEMA prometheus TO admin;
- 创建table
SELECT create_prometheus_table('metrics');
prometheus-postgresql-adapter
./prometheus-postgresql-adapter -pg-port 5432 --pg-user postgres
配置Prometheus
- 在prometheus.yaml中加入
remote_write:
- url: "http://<adapter-address>:9201/write"
remote_read:
- url: "http://<adapter-address>:9201/read"
- 问题本地启动报错
{"caller":"log.go:39","err":"dial tcp [::1]:1921: connect: connection refused","level":"error","ts":"2019-11-20T14:33:22.597Z"}
修改vim /var/lib/pgsql/11/data/pg_hba.conf文件,将认证方式全部改为trust
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
完成后效果
- 本地启动的prometheus-postgresql-adapter日志
[root@localhost PLSQL]# ./prometheus-postgresql-adapter -pg-port 5432 --pg-user postgres
{"caller":"log.go:31","config":"\u0026{remoteTimeout:30000000000 listenAddr::9201 telemetryPath:/metrics pgPrometheusConfig:{host:localhost port:5432 user:postgres password: database:postgres schema: sslMode:disable table:metrics copyTable: maxOpenConns:50 maxIdleConns:10 pgPrometheusNormalize:true pgPrometheusLogSamples:false pgPrometheusChunkInterval:43200000000000 useTimescaleDb:true dbConnectRetries:0 readOnly:false} logLevel:debug haGroupLockID:0 restElection:false prometheusTimeout:-1 electionInterval:5000000000}","level":"info","ts":"2019-11-21T01:02:02.650Z"}
{"caller":"log.go:31","level":"info","msg":"host=localhost port=5432 user=postgres dbname=postgres password='' sslmode=disable connect_timeout=10","ts":"2019-11-21T01:02:02.651Z"}
{"caller":"log.go:35","level":"warn","msg":"No adapter leader election. Group lock id is not set. Possible duplicate write load if running adapter in high-availability mode","ts":"2019-11-21T01:02:02.703Z"}
{"caller":"log.go:31","level":"info","msg":"Starting up...","ts":"2019-11-21T01:02:02.703Z"}
{"addr":":9201","caller":"log.go:31","level":"info","msg":"Listening","ts":"2019-11-21T01:02:02.703Z"}
{"caller":"log.go:27","count":4,"duration":0.092103773,"level":"debug","msg":"Wrote samples","ts":"2019-11-21T01:10:05.197Z"}
{"caller":"log.go:27","count":8,"duration":0.012115927,"level":"debug","msg":"Wrote samples","ts":"2019-11-21T01:10:10.218Z"}
{"caller":"log.go:31","level":"info","msg":"Samples write throughput","samples/sec":4,"ts":"2019-11-21T01:10:10.218Z"}
- postgresql中指标数据
postgres=# select * from metrics;
sample | time |
name | value | labels
--------------------------------------------------------------------------------------------------------------------+----------------------------+---------------
------------------------+-------------+---------------------------------------------------------------
cpu_usage{host="machine1",service="nginx"} 34.600000 1494595898000 | 2017-05-12 09:31:38-04 | cpu_usage
| 34.6 | {"host": "machine1", "service": "nginx"}
up{job="equipment-actuator",instance="10.2.54.59:23504"} 0.000000 1574298596450 | 2019-11-20 20:09:56.45-05 | up
| 0 | {"job": "equipment-actuator", "instance": "10.2.54.59:23504"}
scrape_duration_seconds{job="equipment-actuator",instance="10.2.54.59:23504"} 5.002276 1574298596450 | 2019-11-20 20:09:56.45-05 | scrape_duratio
n_seconds | 5.002276246 | {"job": "equipment-actuator", "instance": "10.2.54.59:23504"}
