Skip to content

Prometheus 指標

OpenClaw 可以透過官方的 diagnostics-prometheus 外掛程式公開診斷指標。它會監聽可信的診斷以及核心發出的閘道穩定性事件,然後在以下位置渲染 Prometheus 文字端點:

GET /api/diagnostics/prometheus

內容類型為 text/plain; version=0.0.4; charset=utf-8,即標準的 Prometheus 格式。

關於追蹤、日誌、OTLP 推送和 OpenTelemetry GenAI 語義屬性,請參閱 OpenTelemetry 匯出

  1. 安裝外掛程式

    Terminal window
    openclaw plugins install clawhub:@openclaw/diagnostics-prometheus
  2. 啟用外掛程式

    {
    plugins: {
    allow: ["diagnostics-prometheus"],
    entries: {
    "diagnostics-prometheus": { enabled: true },
    },
    },
    diagnostics: {
    enabled: true,
    },
    }
  3. 重新啟動 Gateway

    HTTP 路由是在外掛程式啟動時註冊的,因此請在啟用後重新載入。

  4. 抓取受保護的路由

    發送您的 operator 用戶端使用的相同 gateway 驗證:

    Terminal window
    curl -H "Authorization: Bearer $OPENCLAW_GATEWAY_TOKEN" \
    http://127.0.0.1:18789/api/diagnostics/prometheus
  5. 連接 Prometheus

    prometheus.yml
    scrape_configs:
    - job_name: openclaw
    scrape_interval: 30s
    metrics_path: /api/diagnostics/prometheus
    authorization:
    credentials_file: /etc/prometheus/openclaw-gateway-token
    static_configs:
    - targets: ["openclaw-gateway:18789"]

指標類型標籤
openclaw_run_completed_totalcounterchannelmodeloutcomeprovidertrigger
openclaw_run_duration_secondshistogramchannel, model, outcome, provider, trigger
openclaw_model_call_totalcounterapi, error_category, model, outcome, provider, transport
openclaw_model_call_duration_seconds直方圖api, error_category, model, outcome, provider, transport
openclaw_model_failover_total計數器from_model, from_provider, lane, reason, suspended, to_model, to_provider
openclaw_model_tokens_totalcounteragent, channel, model, provider, token_type
openclaw_gen_ai_client_token_usagehistogrammodel, provider, token_type
openclaw_model_cost_usd_total計數器agent, channel, model, provider
openclaw_skill_used_total計數器activation, agent, skill, source
openclaw_tool_execution_total計數器error_category, outcome, params_kind, tool, tool_owner, tool_source
openclaw_tool_execution_duration_seconds直方圖error_category, outcome, params_kind, tool, tool_owner, tool_source
openclaw_tool_execution_blocked_total計數器denied_reason, params_kind, tool, tool_owner, tool_source
openclaw_harness_run_total計數器channel, error_category, harness, model, outcome, phase, plugin, provider
openclaw_harness_run_duration_seconds直方圖channel, error_category, harness, model, outcome, phase, plugin, provider
openclaw_webhook_received_totalcounterchannel, webhook
openclaw_webhook_error_total計數器channel, webhook
openclaw_webhook_duration_seconds直方圖channel, webhook
openclaw_message_received_total計數器channel, source
openclaw_message_dispatch_started_total計數器channel, source
openclaw_message_dispatch_completed_totalcounterchannel, outcome, reason, source
openclaw_message_dispatch_duration_seconds直方圖channel, outcome, reason, source
openclaw_message_processed_total計數器channel, outcome, reason
openclaw_message_processed_duration_secondshistogramchannel, outcome, reason
openclaw_message_delivery_started_total計數器channel, delivery_kind
openclaw_message_delivery_total計數器channel, delivery_kind, error_category, outcome
openclaw_message_delivery_duration_secondshistogramchannel, delivery_kind, error_category, outcome
openclaw_talk_event_totalcounterbrain, event_type, mode, provider, transport
openclaw_talk_event_duration_seconds直方圖brain, event_type, mode, provider, transport
openclaw_talk_audio_bytes直方圖brain, event_type, mode, provider, transport
openclaw_queue_lane_size儀表lane
openclaw_queue_lane_wait_secondshistogramlane
openclaw_session_state_total計數器reason, state
openclaw_session_queue_depth儀表state
openclaw_session_turn_created_totalcounteragent, channel, trigger
openclaw_session_stuck_totalcounterreason, state
openclaw_session_stuck_age_seconds直方圖reason, state
openclaw_session_recovery_total計數器action, active_work_kind, state, status
openclaw_session_recovery_age_seconds直方圖action, active_work_kind, state, status
openclaw_liveness_warning_total計數器reason
openclaw_liveness_sessions儀表state
openclaw_liveness_event_loop_delay_p99_seconds直方圖reason
openclaw_liveness_event_loop_delay_max_seconds直方圖reason
openclaw_liveness_event_loop_utilization_ratio直方圖reason
openclaw_liveness_cpu_core_ratio直方圖reason
openclaw_payload_large_total計數器action, channel, plugin, reason, surface
openclaw_payload_large_bytes直方圖action, channel, plugin, reason, surface
openclaw_memory_bytes儀表kind
openclaw_memory_rss_bytes直方圖
openclaw_memory_pressure_total計數器level, reason
openclaw_telemetry_exporter_total計數器exporter, reason, signal, status
openclaw_prometheus_series_dropped_total計數器
Bounded, low-cardinality labels

Prometheus 標籤保持有界且基數較低。匯出器不會發出原始診斷標識符,例如 runIdsessionKeysessionIdcallIdtoolCallId、訊息 ID、聊天 ID 或提供者請求 ID。

標籤值會被編輯,且必須符合 OpenClaw 的低基數字元策略。未通過該策略的值會根據指標類型替換為 unknownothernone。看起來像範圍代理工作階段金鑰的標籤也會替換為 unknown

Series cap and overflow accounting

匯出器會將記憶體中保留的時間序列上限設為 2048 個,此上限合併計算了計數器、儀表和直方圖。超過此上限的新序列會被丟棄,並且 openclaw_prometheus_series_dropped_total 每次都會加一。

請將此計數器視為上游屬性正在洩漏高基數值的強烈訊號。匯出器永遠不會自動取消上限;如果數值上升,請修復來源而不是停用上限。

What never appears in Prometheus output
  • 提示詞文字、回應文字、工具輸入、工具輸出、系統提示詞
  • 對話紀錄、音訊負載、通話 ID、房間 ID、移交權杖、輪次 ID 和原始工作階段 ID
  • 原始供應商請求 ID(僅在 spans 上使用有界雜湊,如適用——從不在 metrics 上)
  • 工作階段金鑰和工作階段 ID
  • 主機名稱、檔案路徑、秘密值
# Tokens per minute, split by provider
sum by (provider) (rate(openclaw_model_tokens_total[1m]))
# Spend (USD) over the last hour, by model
sum by (model) (increase(openclaw_model_cost_usd_total[1h]))
# 95th percentile model run duration
histogram_quantile(
0.95,
sum by (le, provider, model)
(rate(openclaw_run_duration_seconds_bucket[5m]))
)
# Queue wait time SLO (95p under 2s)
histogram_quantile(
0.95,
sum by (le, lane) (rate(openclaw_queue_lane_wait_seconds_bucket[5m]))
) < 2
# Skill usage, split by bounded source
sum by (skill, source) (increase(openclaw_skill_used_total[24h]))
# Dropped Prometheus series (cardinality alarm)
increase(openclaw_prometheus_series_dropped_total[15m]) > 0

選擇 Prometheus 和 OpenTelemetry 匯出

Section titled “選擇 Prometheus 和 OpenTelemetry 匯出”

OpenClaw 獨立支援這兩個介面。您可以執行其中一個、兩者都執行,或都不執行。

  • Pull 模型:Prometheus 抓取 /api/diagnostics/prometheus
  • 不需要外部收集器。
  • 透過一般的 Gateway 認證進行驗證。
  • 介面僅包含指標(沒有追蹤或日誌)。
  • 最適合已標準化使用 Prometheus + Grafana 的技術堆疊。
回應主體空白
  • 檢查組態中的 diagnostics.enabled: true
  • 確認外掛程式已啟用並已使用 openclaw plugins list --enabled 載入。
  • 產生一些流量;計數器和直方圖僅在至少發生一個事件後才會輸出行。
401 / 未授權

此端點需要 Gateway 操作員範圍(auth: "gateway" 搭配 gatewayRuntimeScopeSurface: "trusted-operator")。使用 Prometheus 用於任何其他 Gateway 操作員路由的相同權杖或密碼。沒有公開的未驗證模式。

`openclaw_prometheus_series_dropped_total` 正在上升

有新屬性超出了 2048 個序列的上限。檢查最近的指標是否有意外的高基數標籤,並從來源修正它。匯出器會故意捨棄新序列,而不是靜默地重寫標籤。

Prometheus 在重新啟動後顯示過時序列

此外掛程式僅在記憶體中保存狀態。在 Gateway 重新啟動後,計數器會重設為零,儀表會在下次回報的值重新啟動。使用 PromQL 的 rate()increase() 來乾淨地處理重設。