用一套 Compose 部署完整的主机/容器监控：Prometheus 采集存储 + Grafana 展示 + node-exporter 主机指标，所有配置和数据 bind mount 到宿主机，密码走 .env，监控端口默认只绑回环，远程通过 SSH 隧道访问。

环境：一台 2C/4G 以上 Linux 主机，已安装 Docker Engine 与 Compose v2。

一、目录结构

/opt/monitoring/
├── docker-compose.yml
├── .env                      # 600，放 Grafana 管理员密码
├── prometheus/
│   ├── prometheus.yml
│   ├── rules/
│   │   └── alerts.yml
│   └── data/                 # chown 65534:65534（nobody）
└── grafana/
    ├── provisioning/
    │   ├── datasources/datasource.yml
    │   └── dashboards/dashboards.yml
    ├── dashboards/          # 放 JSON 看板，自动加载
    └── data/                # chown 472:472（grafana 镜像 uid）

sudo mkdir -p /opt/monitoring/{prometheus/{rules,data},grafana/{provisioning/datasources,provisioning/dashboards,dashboards,data}}
sudo chown -R 65534:65534 /opt/monitoring/prometheus/data
sudo chown -R 472:472 /opt/monitoring/grafana/data

二、.env（权限 600）

umask 077
cat > /opt/monitoring/.env <<'EOF'
GF_ADMIN_PASSWORD=替换为强密码
TZ=Asia/Shanghai
EOF
chmod 600 /opt/monitoring/.env

生成强密码：openssl rand -base64 24。

三、docker-compose.yml

name: monitoring

x-logging: &default-logging
  driver: json-file
  options:
    max-size: "20m"
    max-file: "3"

services:
  prometheus:
    image: prom/prometheus:v2.54.1
    container_name: prometheus
    restart: unless-stopped
    logging: *default-logging
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.path=/prometheus
      - --storage.tsdb.retention.time=30d
      - --web.enable-lifecycle
    ports:
      - "127.0.0.1:9090:9090"      # 只绑回环，走 SSH 隧道访问
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ./prometheus/rules:/etc/prometheus/rules:ro
      - ./prometheus/data:/prometheus
    networks:
      - monitoring-net
    extra_hosts:
      - "host.docker.internal:host-gateway"
    security_opt:
      - no-new-privileges:true
    mem_limit: 1g
    healthcheck:
      test: ["CMD", "wget", "-q", "-O", "-", "http://localhost:9090/-/healthy"]
      interval: 15s
      timeout: 5s
      retries: 5

  grafana:
    image: grafana/grafana:11.2.0
    container_name: grafana
    restart: unless-stopped
    logging: *default-logging
    environment:
      GF_SECURITY_ADMIN_USER: admin
      GF_SECURITY_ADMIN_PASSWORD: ${GF_ADMIN_PASSWORD}
      GF_USERS_ALLOW_SIGN_UP: "false"
      GF_USERS_ALLOW_ORG_CREATE: "false"
      GF_AUTH_ANONYMOUS_ENABLED: "false"
      GF_SECURITY_COOKIE_SECURE: "false"   # 套上 HTTPS 后改成 true
      GF_USERS_DEFAULT_THEME: light
      TZ: ${TZ}
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - ./grafana/data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning:ro
      - ./grafana/dashboards:/var/lib/grafana/dashboards:ro
    networks:
      - monitoring-net
    depends_on:
      - prometheus
    security_opt:
      - no-new-privileges:true
    mem_limit: 512m
    healthcheck:
      test: ["CMD-SHELL", "curl -fsS http://localhost:3000/api/health || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 5

  node-exporter:
    image: prom/node-exporter:v1.8.2
    container_name: node-exporter
    restart: unless-stopped
    logging: *default-logging
    network_mode: host          # 直接读宿主指标
    pid: host
    command:
      - --path.procfs=/host/proc
      - --path.sysfs=/host/sys
      - --path.rootfs=/rootfs
      - --collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc|var/lib/docker)($$|/)
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    mem_limit: 256m

networks:
  monitoring-net:
    name: monitoring-net
    driver: bridge

要点：

• 镜像全部锁版本，不用 latest；国内拉不动时走加速器或 docker pull docker.m.daocloud.io/prom/prometheus:v2.54.1 再 retag。
• Prometheus/Grafana 端口只绑 127.0.0.1，不对公网暴露。
• Prometheus 容器以 nobody 运行，数据目录宿主 chown 65534。
• node-exporter 用 host 网络，直接监听宿主 9100，Prometheus 通过 host.docker.internal 采它。

四、Prometheus 配置

prometheus/prometheus.yml：

global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:
  - /etc/prometheus/rules/*.yml

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['prometheus:9090']

  - job_name: 'node'
    static_configs:
      - targets: ['host.docker.internal:9100']
        labels:
          { instance: 'host' }

一个示例告警规则 prometheus/rules/alerts.yml（主机磁盘/内存）：

groups:
  - name: host-basic
    rules:
      - alert: HostDiskWillFillIn4h
        expr: predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"}[1h], 4*3600) < 0
        for: 10m
        labels: { severity: warning }
        annotations:
          summary: "磁盘预计 4 小时内写满：{{ $labels.mountpoint }}"

      - alert: HostHighMemoryUsage
        expr: 100 - (100 * node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) > 90
        for: 5m
        labels: { severity: warning }
        annotations:
          summary: "内存使用率超过 90%（当前 {{ $value | printf \"%.1f\" }}%）"

五、Grafana 自动 provisioning

数据源 grafana/provisioning/datasources/datasource.yml：

apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    editable: false

看板 provider grafana/provisioning/dashboards/dashboards.yml：

apiVersion: 1
providers:
  - name: 'default'
    orgId: 1
    folder: ''
    type: file
    allowUiUpdates: true
    updateIntervalSeconds: 30
    options:
      path: /var/lib/grafana/dashboards

把社区看板 JSON（如 Node Exporter Full，ID 1860）放进 grafana/dashboards/ 即自动加载，无需手工导入。

六、启动与验证

cd /opt/monitoring
docker compose config -q          # 语法检查
docker compose up -d
docker compose ps                 # 三个服务应都 Up，前两个 healthy

验证采集目标：

curl -s http://127.0.0.1:9090/api/v1/targets | grep -o '"health":"[^"]*"'

本机临时开隧道访问（在你自己的电脑执行）：

ssh -L 3000:127.0.0.1:3000 -L 9090:127.0.0.1:9090 user@203.0.113.10

然后浏览器开 http://127.0.0.1:3000，用 .env 里的密码登录；Prometheus 为 http://127.0.0.1:9090。

如果确实要对公网开放 Grafana，建议只放在 Nginx/Caddy 反代后面上 HTTPS，并把 GF_SECURITY_COOKIE_SECURE 改 true，同时用云安全组限制来源 IP，不要把 3000 端口裸奔。

七、可选扩展

• 容器指标：Docker 29 起 containerd 镜像存储让经典 cAdvisor 拿不到容器层，可改用 otel/opentelemetry-collector-contrib 的 docker_stats receiver（走 Docker API），在 8889 暴露 Prometheus 指标后加入 scrape。
• MySQL / Redis 指标：加 prom/mysqld-exporter、oliver006/redis_exporter，让它们接入被监控服务所在的 compose 外部网络即可，不需要发宿主端口。
• 告警通知：再加 alertmanager，或用一个小中转服务把 Alertmanager v2 webhook 转成飞书/钉钉/企业微信消息。

八、常见问题