AB Test Framework

通过给定测试提示词对两个模型做 A/B 测试对比，返回状态、详情、优胜模型和置信度；声明输入校验、最小权限与审计日志等安全措施，但未给出具体统计实现和判定标准。

热度 87.60

最近更新 2026-05-17 12:49:58

分类：数据依赖：代码执行 / 内部调用大模型应用场景：研究 / 报告撰写

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add ab-test-framework

Metadata

name: A/B Testing Framework
version: 1.0.0
description: Compare models with A/B testing for selection
author: OpenClaw SysAdmin Community
tags: [20,"sysadmin","manual"]
trigger: {"type":"manual","schedule":"","enabled":true}
input: {"model_a":{"type":"string","description":"First model","required":true},"model_b":{"type":"string","description":"Second model","required":true},"test_prompts":{"type":"array","description":"Test prompts","required":true}}
output: {"status":"string","details":"object","winner":"string","confidence":"number"}
dependencies: ["openclaw/llm","stats-library"]
security: ["A/B test security per Category 8; prevent test manipulation","Validate all inputs per Category 8","Use least privilege principle","Log all operations for audit"]

Compare models with A/B testing for selection

This skill is derived from 20. Testing & Quality Assurance of the OpenClaw Agent Mastery Index v4.1.

Sub-heading: A/B Testing Frameworks for Model Selection

Complexity: high

Name	Type	Required	Description
`model_a`	string	Yes	First model
`model_b`	string	Yes	Second model
`test_prompts`	array	Yes	Test prompts

{
  "status": <string>,
  "details": <object>,
  "winner": <string>,
  "confidence": <number>
}

const result = await openclaw.skill.run('ab-test-framework', {
  model_a: "value",
  model_b: "value",
  test_prompts: 123
});

const result = await openclaw.skill.run('ab-test-framework', {
  model_a: "value",
  model_b: "value",
  test_prompts: []
});

A/B test security per Category 8; prevent test manipulation

Issue	Cause	Solution
Permission denied	Insufficient privileges	Check file/directory permissions
Invalid input	Malformed parameters	Validate input format
Dependency missing	Required module not installed	Run `npm install`

Enable debug logging:

openclaw.logger.setLevel('debug');
const result = await openclaw.skill.run('ab-test-framework', { ... });