Pilot Heartbeat Monitor

核心功能：通过周期性心跳监控代理健康，基于超时机制检测故障。典型场景：集群成员不可达检测、故障自动转移、负载均衡或选主的健康监控。需依赖pilot-protocol技能、pilotctl二进制及运行的daemon，使用pilotctl命令发送心跳、排查故障，输出故障代理列表，可触发任务重分配等恢复动作。

teoslayer

热度 40.60

最近更新 2026-05-11 17:41:32

分类：自动化依赖：代码执行 / 第三方 API应用场景：health-monitoring / failure-detection / 自动化脚本

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add pilot-heartbeat-monitor

Metadata

name: pilot-heartbeat-monitor
description: Detect agent failures and trigger automatic task redistribution or re-election. Use this skill when: 1. You need to detect when swarm members become unreachable 2. You want to trigger failover actions on agent failure 3. You need health monitoring for load balancing or leader election Do NOT use this skill when: - Agents can safely fail without recovery (fire-and-forget tasks) - Network partitions are rare and acceptable (use simpler ping checks)
tags: ["pilot-protocol","health-monitoring","failure-detection","heartbeat"]
license: AGPL-3.0
compatibility: Requires pilot-protocol skill and pilotctl binary on PATH. The daemon must be running (pilotctl daemon start).
metadata: {"author":"vulture-labs","version":"1.0","openclaw":{"requires":{"bins":["pilotctl"]},"homepage":"https://pilotprotocol.network"}}
allowed-tools: ["Bash"]

pilot-heartbeat-monitor

Monitor agent health via periodic heartbeats and detect failures using timeout-based detection.

Commands

Send heartbeat to peers

pilotctl --json publish "registry-hostname" "heartbeat:$SWARM_NAME" \
  --data "{\"agent\":\"$AGENT_ID\",\"timestamp\":\"$(date -u +%s)\",\"status\":\"alive\"}"

Detect failed agents

TIMEOUT=30
CURRENT_TIME=$(date -u +%s)

FAILED_AGENTS=$(pilotctl --json inbox \
  | jq --arg now "$CURRENT_TIME" --arg timeout "$TIMEOUT" \
    '[.messages[] | select(.topic == "heartbeat:'$SWARM_NAME'") | {agent: .payload.agent, last_seen: .payload.timestamp}]
    | group_by(.agent)
    | map(select(($now | tonumber) - (map(.last_seen) | max) > ($timeout | tonumber)))
    | .[].agent')

Verify failure with direct ping

for agent in $FAILED_AGENTS; do
  AGENT_ADDR=$(pilotctl --json peers | jq -r '.[] | select(.node_id == "'$agent'") | .address')

  PING_RESULT=$(pilotctl --json ping "$AGENT_ADDR" --count 3 --timeout 2s)
  LOSS=$(echo "$PING_RESULT" | jq -r '.packet_loss_pct')

  if [ "$LOSS" = "100" ]; then
    echo "Agent $agent CONFIRMED DOWN"
  fi
done

Workflow Example

Health monitor for worker pool with automatic task redistribution:

#!/bin/bash
SWARM_NAME="worker-pool"
HEARTBEAT_INTERVAL=5
FAILURE_TIMEOUT=15
REGISTRY_HOST="registry.example.com"

# Background: Send own heartbeats
(
  while true; do
    pilotctl --json publish "$REGISTRY_HOST" "heartbeat:$SWARM_NAME" \
      --data "{\"agent\":\"$AGENT_ID\",\"timestamp\":\"$(date -u +%s)\"}"
    sleep $HEARTBEAT_INTERVAL
  done
) &

# Monitor peer heartbeats
while true; do
  CURRENT_TIME=$(date -u +%s)

  # Detect timeouts and trigger recovery
  # ...

  sleep $HEARTBEAT_INTERVAL
done

Dependencies

Requires pilot-protocol skill, jq, and bc.