360 Secure SkillHub
360 安全技能中心

Provider Probe

核心功能：通过分层测试（/models检查、兼容性验证、稳定性测试等）判断OpenAI兼容接口是单模型端点还是多模型聚合池，给出三级信任评级。典型场景：验证声称模型真伪、检查隐藏模型池、对比同模型提供商、生成路由用信任报告。输入支持配置文件或自定义baseURL+密钥+模型ID，输出含测试进展、阻塞点、建议及最终信任判断与路由推荐。

andyrenxu7255
andyrenxu7255
热度 47.92
最近更新 2026-05-17 08:21:48
分类：安全依赖：代码执行 / 第三方 API / 读写文件应用场景：报告撰写 / API 集成 / 数据处理

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add provider-probe
Metadata
name
provider-probe
description
Probe and verify whether an OpenAI-compatible baseURL is a real single-model endpoint or a multi-model aggregation pool. Use when auditing model providers, checking /models output, comparing completions vs responses support, validating claimed model IDs like gpt-5.4, or producing a provider trust/stability report for OpenClaw routing decisions.

Provider Probe

Use this skill to investigate model providers behind OpenAI-compatible base URLs.

When to use

Trigger this skill when the user asks to:

  • verify whether a provider's claimed model is real
  • inspect a baseURL for hidden/mixed model pools
  • compare multiple providers for the same claimed model
  • determine whether a provider is better suited as primary or fallback
  • create a trust/stability report for model routing

Core method

Always use a layered evidence approach:

  1. Read provider config or ask for baseURL + apiKey + claimed model id.
  2. Call /models and inspect whether the returned pool contains mixed vendors or suspicious aliases.
  3. Check metadata like owned_by, model naming conventions, and whether one baseURL exposes many unrelated model families.
  4. Probe both /responses and /chat/completions with minimal prompts.
  5. Run short capability tests and repeated stability tests.
  6. Summarize with a confidence rating rather than absolute certainty.

Confidence labels

  • High confidence real / most likely genuine: stable, coherent endpoint behavior, believable output structure, low ambiguity.
  • Medium confidence / likely routed or wrapped: works, but signs suggest aggregation, aliasing, or proxy adaptation.
  • Low confidence / unusable now: 404, repeated timeout, incompatible shape, or too little evidence.

Output contract

Always report:

  • 当前做到哪了 / what was tested
  • 当前阻塞点 / what remains uncertain
  • 下一步动作 / recommended next step

For final results, include:

  1. Config facts
  2. /models findings
  3. Endpoint compatibility findings
  4. Repeated stability findings
  5. Capability/format findings
  6. Final trust judgment
  7. Recommendation: primary / fallback / avoid

Tooling

Prefer the bundled script for deterministic testing:

  • scripts/provider_probe.py

Usage:

python3 scripts/provider_probe.py --config /root/.openclaw/openclaw.json --providers ypemc omgteam vpsai --model gpt-5.4

Or probe a custom URL directly:

python3 scripts/provider_probe.py --base-url https://example.com/v1 --api-key sk-xxx --model gpt-5.4

Interpretation heuristics

Treat a provider as a likely aggregation pool when several of these appear together:

  • /models returns many unrelated model families
  • owned_by values are mixed or inconsistent
  • the claimed model id looks like a routing alias rather than a canonical model id
  • /responses and /chat/completions compatibility is uneven or surprising
  • behavior is stable enough to work but not coherent enough to look like a single official upstream

Files

  • Reference checklist: references/provider-probe-checklist.md
  • Probe script: scripts/provider_probe.py