360 Secure SkillHub
360 安全技能中心

Jev AI Model Evaluator

该技能面向具体AI任务开展模型评估，默认文本交互，可选配只读确定性MCP工具（不调用模型、不消耗额度、不访问隐私数据）。支持基于透明加权规则设计评估方案、构建固定测试集，对比候选模型的成本、延迟、质量表现，核验厂商宣传声明，适用于模型选型、API对比、基准设计等场景，输出可溯源的结构化评估结论，避免过度泛化小样本测试结果。

gpt-img-2
gpt-img-2
热度 16.23
最近更新 2026-10-09 22:19:45
分类：开发依赖：纯提示词 / MCP 服务应用场景：API 集成 / 数据处理 / 报告撰写 / 研究

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add jev-ai-model-evaluator
Metadata
name
jev-ai-model-evaluator
description
Plan reproducible AI model evaluations, compare candidates with transparent criteria, and audit model claims. Use for model selection, benchmark design, API model comparisons, playground testing, cost-latency-quality tradeoffs, and Jev AI Model workflows; do not generalize small tests or provider claims into universal superiority.
metadata
{"openclaw":{"homepage":"https://jevaimodel.org/en/docs","version":"1.0.0"}}

Jev AI Model Evaluator

Help the user evaluate AI models for a concrete task. This Skill is text-only by default. Its optional MCP is deterministic and read-only: it does not call models, access accounts, consume credits, fetch private data, or select a model without inspectable user-supplied evidence.

Define the evaluation

Specify the real task, expected output, target language or region, deployment constraints, acceptable latency and cost, risk level, weighted criteria, and must-pass checks. Keep a frozen test set and identical settings across candidates.

Include ordinary, edge, ambiguous, adversarial, and failure cases. Record model version, parameters, date, latency, errors, refusals, and observed outputs. For high-risk work, require domain-expert review and explicit must-pass safety checks.

Interpret results carefully

  • Keep provider claims, third-party benchmarks, reproducible tests, and user reports separate.
  • A benchmark score only supports the benchmark's task, dataset, settings, and date.
  • Do not hide must-pass failures inside an average score.
  • Verify current versions, prices, limits, regions, latency, and availability.
  • Prefer a provisional result over a manufactured winner when evidence is incomplete.

Use the optional MCP

  • design_model_evaluation builds a weighted plan and dataset checklist.
  • rank_model_candidates calculates transparent scores from user-supplied 0–5 ratings and preserves must-pass failures.
  • audit_model_claims checks sources, dates, evidence types, and unsupported absolutes.
  • get_jev_ai_model_resources returns canonical public Jev AI Model pages.

Connect the stdio server with:

openclaw mcp add jev-ai-model \
  --command npx \
  --arg -y \
  --arg github:gpt-img-2/jev-ai-model-evaluation-mcp \
  --include 'design_model_evaluation,rank_model_candidates,audit_model_claims,get_jev_ai_model_resources'

Then run openclaw mcp doctor jev-ai-model --probe. If the installed release lacks that command group, add npx -y github:gpt-img-2/jev-ai-model-evaluation-mcp as a Stdio server in MCP settings. Never imply the MCP or a model ran when it did not.

Return an inspectable result

Return Task, Dataset, Settings, Criteria, Must-pass checks, Candidates, Evidence, Scores, Failures, Recommendation, and Next verification.

Canonical resources