PMM MySQL 巡检日报 2026-10-26
- 数据源:PMM (192.168.8.52:8443),经 MCP (192.168.8.75:8200) 直连获取(pmm_status,秒级采样快照)
- 数据生成时间:2026-10-26 02:00:12 UTC(北京时间 2026-10-26 10:00:12)
- 报告生成时间:2026-10-26(Asia/Shanghai)
概览
| 指标 | 值 |
|---|
| 实例总数 | 22 |
| up 数 | 22(100%) |
| alert 数 | 1(critical 1 / warning 0) |
Critical(实例 down / 复制中断)
| 服务 | 问题 | 详情 |
|---|
| t591-dealhub | replication_stopped | IO=False, SQL=True(IO 线程停止,SQL 线程仍在运行) |
建议:经 matched_alias staging/t591-dealhub(mysql-57-iomesh-t591-dealhub.database.svc.cluster.local:3306)连接后用 SHOW REPLICA STATUS 检查 Last_IO_Error,确认主库可达性与复制账号权限后重启 IO 线程。深挖链路:connect → show_active_connections / show_lock_waits。
Warning(延迟>60s / 连接使用率≥80% / 慢查询≥1/s)
alerts 数组中无 warning 项,已按阈值扫描全部 22 实例 data 复核:
- 复制延迟 > 60s:无(所有从库 seconds_behind_master 均为 0s;tw591-slave-lag 当前 sql_delay=0,未触发延迟从库标注)
- 连接使用率 ≥ 80%:无(最高 td_master 3.5%,其余均 ≤ 2.3%)
- 慢查询 ≥ 1/s:无(仅 t8_mysql slow_qps=0.01/s,其余均为 0)
全实例明细
| service_name | up | uptime(天) | 连接 used/max(pct%) | threads_running | qps | slow_qps | BP命中率% | 复制延迟s | matched_alias |
|---|
| 100-master | ✅ | 87.9 | 21/8192 (0.3%) | 5 | 9.68 | 0.00 | 99.8 | 0 | staging/100 |
| 8891-master | ✅ | 563.9 | 18/2000 (0.9%) | 3 | 12.05 | 0.00 | 100.0 | - | staging/8891 |
| 8891-slave | ✅ | 440.8 | 3/2000 (0.1%) | 1 | 4.33 | 0.00 | 100.0 | 0 | staging/8891-slave |
| TD-slave | ✅ | 465.0 | 4/2048 (0.2%) | 3 | 4.32 | 0.00 | 99.8 | 0 | staging/TD-slave2 |
| TD-slave-51 | ✅ | 465.0 | 4/2048 (0.2%) | 2 | 4.39 | 0.00 | 99.7 | 0 | staging/TD-slave |
| addcn-bee | ✅ | 74.8 | 2/2048 (0.1%) | 2 | 5.70 | 0.00 | 100.0 | - | staging/addcn-bee |
| hk8591 | ✅ | 221.6 | 10/15000 (0.1%) | 5 | 13.62 | 0.00 | 100.0 | 0 | staging/hk8591 |
| hk8591-slave | ✅ | 445.7 | 5/15000 (0.0%) | 4 | 4.46 | 0.00 | 100.0 | 0 | staging/hk8591-slave |
| hongyan | ✅ | 55.8 | 5/2048 (0.2%) | 3 | 4.68 | 0.00 | 100.0 | - | staging/hongyan |
| mcdb-master | ✅ | 139.4 | 43/4096 (1.0%) | 4 | 18.65 | 0.00 | 100.0 | 0 | staging/mcdb |
| mcdb-slave | ✅ | 230.8 | 5/4096 (0.1%) | 4 | 4.58 | 0.00 | 100.0 | 0 | staging/mcdb-slave |
| t1_mysql | ✅ | 87.9 | 14/9190 (0.2%) | 4 | 5.49 | 0.00 | 99.9 | 0 | staging/100-slave |
| t5-ai | ✅ | 74.8 | 10/2048 (0.5%) | 3 | 4.78 | 0.00 | 100.0 | - | staging/t5-ai |
| t5-gateway | ✅ | 74.8 | 5/2048 (0.2%) | 2 | 4.55 | 0.00 | 100.0 | - | staging/t5-gateway |
| t591-dealhub | ✅ | 74.8 | 4/2048 (0.2%) | 2 | 4.78 | 0.00 | 100.0 | ⚠️ IO线程停 | staging/t591-dealhub |
| t8-ai | ✅ | 74.8 | 3/2048 (0.1%) | 3 | 4.77 | 0.00 | 100.0 | - | staging/t8-ai |
| t8-antifraud | ✅ | 18.0 | 48/2048 (2.3%) | 3 | 4.67 | 0.00 | 91.8 | - | staging/t8-antifraud |
| t8_mysql | ✅ | 107.6 | 224/15000 (1.5%) | 18 | 121.17 | 0.01 | 99.9 | 0 | staging/tw8591 |
| td_master | ✅ | 716.0 | 71/2048 (3.5%) | 5 | 159.27 | 0.00 | 100.0 | - | staging/TD |
| tw591-slave | ✅ | 465.0 | 4/2048 (0.2%) | 4 | 4.03 | 0.00 | 100.0 | - | staging/tw591-slave |
| tw591-slave-lag | ✅ | 18.0 | 3/2048 (0.1%) | 2 | 4.40 | 0.00 | 100.0 | 0 | staging/tw591-slave2 |
| tw8591-slave | ✅ | 445.7 | 5/15000 (0.0%) | 3 | 4.40 | 0.00 | 99.9 | 0 | staging/tw8591-slave |
注:复制延迟列为 "-" 表示该实例非从库(replication 为 null);t591-dealhub 的 ⚠️ 见 Critical 部分。t8-antifraud BP 命中率 91.8% 略低于其余实例(多为 99.7%+),该实例 uptime 仅 18 天,可能为近期重启后缓冲池尚未完全预热,暂不构成告警,下轮持续观察。
QPS Top 3
| 排名 | service_name | qps | threads_running |
|---|
| 1 | td_master | 159.27 | 5 |
| 2 | t8_mysql | 121.17 | 18 |
| 3 | mcdb-master | 18.65 | 4 |
数据源 PMM(192.168.8.52:8443),经 MCP(8.75:8200) 直连获取,下轮明日 09:50。