PMM MySQL 巡检日报 2026-10-23
- 数据源：PMM (192.168.8.52:8443) VictoriaMetrics 秒级采样，经 MCP (192.168.8.75:8200)
pmm_status 直连获取
- 数据时间：2026-10-23 02:00:13（UTC）｜报告生成时间：2026-10-23 10:00 (Asia/Shanghai)
- 巡检性质：只读快照，未对数据库做任何写/kill/重启操作
概览
Critical（实例 down / 复制中断）
| 实例 | 问题 | 详情 |
|---|
| t591-dealhub | replication_stopped | IO=False, SQL=True（IO 线程停止，复制中断） |
深挖链路：staging/t591-dealhub (mysql-57-iomesh-t591-dealhub.database.svc.cluster.local:3306) → connect → SHOW SLAVE STATUS / show_active_connections。
Warning（阈值扫描复核）
按阈值复核全量 data（alerts 未覆盖部分）：延迟>60s、连接使用率≥80%、慢查询≥1/s —— 无。
- 复制延迟：最高为 tw591-slave-lag 1s（延迟从库，设计如此），其余均为 0s
- 连接使用率：最高 td_master 3.1%，无 ≥80% 实例
- 慢查询：最高 t8_mysql 0.01/s，无 ≥1/s 实例
- 附注：tw591-slave-lag 历史连接峰值 max_used=2049 已触及 max=2048（曾打满），当前 used=4 正常；t8-antifraud BP 命中率 92.8% 偏低（重启后仅 15 天，属预热中，暂不告警）
全实例明细
| service_name | up | uptime(天) | 连接 used/max(pct%) | threads_running | qps | slow_qps | BP命中率% | 复制延迟s | matched_alias |
|---|
| 100-master | OK | 84.9 | 18/8192(0.2%) | 7 | 10.83 | 0 | 100 | 0.0 | staging/100 |
| 8891-master | OK | 560.9 | 16/2000(0.8%) | 3 | 7.14 | 0 | 100 | - | staging/8891 |
| 8891-slave | OK | 437.8 | 3/2000(0.1%) | 1 | 4.33 | 0 | 100 | 0.0 | staging/8891-slave |
| TD-slave | OK | 461.9 | 4/2048(0.2%) | 3 | 4.32 | 0 | 99.8 | 0.0 | staging/TD-slave2 |
| TD-slave-51 | OK | 461.9 | 4/2048(0.2%) | 4 | 4.39 | 0 | 99.8 | 0.0 | staging/TD-slave |
| addcn-bee | OK | 71.8 | 2/2048(0.1%) | 1 | 5.67 | 0 | 100 | - | staging/addcn-bee |
| hk8591 | OK | 218.6 | 13/15000(0.1%) | 5 | 12.97 | 0 | 100 | 0.0 | staging/hk8591 |
| hk8591-slave | OK | 442.7 | 5/15000(0.0%) | 4 | 4.46 | 0 | 100 | 0.0 | staging/hk8591-slave |
| hongyan | OK | 52.8 | 4/2048(0.2%) | 3 | 4.61 | 0 | 100 | - | staging/hongyan |
| mcdb-master | OK | 136.4 | 34/4096(0.8%) | 5 | 13.92 | 0 | 99.9 | 0.0 | staging/mcdb |
| mcdb-slave | OK | 227.8 | 4/4096(0.1%) | 4 | 4.57 | 0 | 100 | 0.0 | staging/mcdb-slave |
| t1_mysql | OK | 84.9 | 11/9190(0.1%) | 4 | 4.95 | 0 | 99.9 | 0.0 | staging/100-slave |
| t5-ai | OK | 71.8 | 10/2048(0.5%) | 3 | 4.79 | 0 | 100 | - | staging/t5-ai |
| t5-gateway | OK | 71.8 | 5/2048(0.2%) | 2 | 4.55 | 0 | 100 | - | staging/t5-gateway |
| t591-dealhub | OK | 71.8 | 4/2048(0.2%) | 2 | 4.78 | 0 | 100 | 0.0 ⚠️IO线程停 | staging/t591-dealhub |
| t8-ai | OK | 71.8 | 4/2048(0.2%) | 3 | 4.77 | 0 | 100 | - | staging/t8-ai |
| t8-antifraud | OK | 15.0 | 45/2048(2.2%) | 4 | 4.65 | 0 | 92.8 | - | staging/t8-antifraud |
| t8_mysql | OK | 104.6 | 217/15000(1.4%) | 17 | 117.17 | 0.01 | 100 | 0.0 | staging/tw8591 |
| td_master | OK | 712.9 | 63/2048(3.1%) | 6 | 153.8 | 0 | 99.9 | - | staging/TD |
| tw591-slave | OK | 461.9 | 4/2048(0.2%) | 3 | 4.22 | 0 | 100 | - | staging/tw591-slave |
| tw591-slave-lag | OK | 15.0 | 4/2048(0.2%) | 2 | 4.62 | 0 | 99.9 | 1.0（延迟从库,设计如此） | staging/tw591-slave2 |
| tw8591-slave | OK | 443.0 | 5/15000(0.0%) | 3 | 4.4 | 0 | 100 | 0.0 | staging/tw8591-slave |
QPS Top 3
| 排名 | 实例 | QPS | threads_running |
|---|
| 1 | td_master | 153.8 | 6 |
| 2 | t8_mysql | 117.17 | 17 |
| 3 | mcdb-master | 13.92 | 5 |
数据源 PMM(192.168.8.52:8443)，经 MCP(8.75:8200) 直连获取，下轮明日 09:50。