Evalpal

该 Skill 用三个 Bash 脚本调用 EvalPal API：触发评测运行并轮询结果、查询运行状态、列出可用评测定义，并以 Markdown 形式输出通过/失败统计与退出码。评测逻辑由外部服务执行，需配置 EVALPAL_API_KEY，依赖 curl 与 jq。

matthewengman

热度 55.59

最近更新 2026-05-11 17:37:03

分类：开发依赖：代码执行 / 第三方 API应用场景：API 集成 / 自动化脚本 / 报告撰写

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add evalpal

Metadata

name: evalpal
description: Run AI agent evaluations via EvalPal — trigger eval runs, check results, and list available evaluations
homepage: https://evalpal.dev
user-invocable: true
requires: {"env":["EVALPAL_API_KEY"],"bins":["curl","jq"]}

EvalPal Skill

Run AI agent evaluations inline. Trigger eval runs, poll for results, and list available evaluation definitions — all from chat.

Prerequisites

Set the following environment variables in your OpenClaw skill configuration:

Variable	Required	Description
`EVALPAL_API_KEY`	Yes	Your EvalPal API key (starts with `sk_`)
`EVALPAL_API_URL`	No	Base URL (defaults to `https://evalpal.dev`)

Get your API key from Settings → API Keys at evalpal.dev.

Commands

`/evalpal run --eval-id <ID>`

Trigger an evaluation run and wait for results.

Usage:

bash scripts/run-eval.sh --eval-id <EVAL_DEFINITION_ID>

What it does:

Triggers a new eval run via the EvalPal API
Polls for completion with exponential backoff (up to 5 minutes)
Fetches and formats results as readable markdown

Example output:

✅ Episode Quality — PASSED (15/16)
├── Test Case tc_001: ✓ PASS
├── Test Case tc_002: ✓ PASS
├── Test Case tc_003: ✗ FAIL
└── 12 more passed...

Run ID: run_abc123 · 16 test cases · 47s

Exit codes: 0 = all passed, 1 = failures or error.

`/evalpal status --run-id <ID>`

Check the current status of a running evaluation.

Usage:

bash scripts/check-status.sh --run-id <RUN_ID>

Example output:

📊 Run Status: run_abc123
Status: running
Started: 2026-03-26T20:00:00Z

`/evalpal list`

List available evaluation definitions across your projects.

Usage:

bash scripts/list-evals.sh [--project-id <PROJECT_ID>]

If --project-id is omitted, lists evals for all projects.

Example output:

📋 Evaluation Definitions

Project: AI Workforce Lab
  abc123  Episode Quality Check
  def456  Factual Accuracy Eval

Project: Customer Support Bot
  ghi789  Response Quality

Error Handling

All scripts handle common error cases:

Scenario	Output	Exit Code
No API key set	`Error: EVALPAL_API_KEY is not set`	1
Invalid API key	`Error: Authentication failed (401)`	1
Eval not found	`Error: Eval definition not found (404)`	1
Rate limited	`Error: Rate limited — retry after Xs (429)`	1
Timeout (5 min)	`Error: Evaluation timed out after 300s`	1
Network error	`Error: Could not reach EvalPal API`	1

Security

The API key is read from EVALPAL_API_KEY environment variable only
Scripts never echo or log the API key
All API calls use HTTPS