360 Secure SkillHub
360 安全技能中心

Llm Provider Forensics

用于核查声称的 LLM 接口或模型 ID：探测目录与最小推理端点，区分协议层结论与疑似模型家族结论，检查重复稳定性、严格格式控制、流式与错误指纹、随机性与跨协议一致性，可对比多个提供方，并按置信度输出可用性判断与运维姿态建议。

andyrenxu7255
andyrenxu7255
热度 48.71
最近更新 2026-05-17 08:11:50
分类：安全依赖：代码执行 / 第三方 API / 读写文件应用场景：研究 / 报告撰写 / API 集成 / 自动化脚本

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add llm-provider-forensics
Metadata
name
llm-provider-forensics
description
Forensically verify what model family or routing layer may actually sit behind a claimed LLM endpoint or model ID. Use when an agent must investigate whether a provider is genuine, proxied, aliased, aggregated, wrapped, or currently unusable across OpenAI-compatible protocol layers, GPT/OpenAI, Anthropic/Claude, Google Gemini, GLM/Zhipu, Qwen/Tongyi, Kimi/Moonshot, MiniMax, DeepSeek, and mixed compatibility gateways. Supports deeper family-fingerprint analysis, long-context tests, structured-output stress, refusal and variance profiling, streaming/error clues, repeated stability checks, and cross-provider comparison reports.

LLM Provider Forensics

Agent-facing forensic skill for identifying what an LLM endpoint most likely is.

Trigger conditions

Use this skill when asked to:

  • verify whether a claimed model is genuine
  • identify which family an endpoint most resembles
  • distinguish focused route vs wrapped route vs aggregation pool
  • compare multiple providers claiming to expose the same model
  • evaluate primary/fallback/avoid decisions
  • deeply audit suspicious gateways for GPT / Claude / Gemini / GLM / Qwen / Kimi / MiniMax / DeepSeek behavior

Core rule

Do not output false certainty. Produce a confidence-based operational judgment.

Coverage

Families:

  • OpenAI-compatible protocol layer
  • GPT / OpenAI-style
  • Claude / Anthropic-style
  • Gemini / Google-style
  • GLM / Zhipu-style
  • Qwen / Tongyi-style
  • Kimi / Moonshot-style
  • MiniMax-style
  • DeepSeek-style
  • mixed aggregation pool / compatibility gateway

Dimensions:

  • catalog topology
  • protocol compatibility
  • response schema shape
  • repeated stability
  • strict formatting control
  • family fingerprinting
  • long-context retention
  • structured-output stress
  • refusal/safety style
  • randomness / variance profile
  • streaming / error fingerprints
  • cross-protocol consistency

Current implementation note:

  • openai-compatible now means protocol layer only, not GPT-family proof.
  • The deepest automatic suite is strongest for OpenAI-compatible / mixed gateway providers.
  • Anthropic-native and Gemini-native routes currently have solid protocol/family checks, plus native deep tests, but protocol success alone must not be read as family proof.
  • Treat all family conclusions as confidence-based and inspect references before overclaiming.

Investigation workflow

  1. Identify likely protocol family or families.
  2. Probe catalog/list endpoints when available.
  3. Probe minimal inference endpoints for each plausible protocol family.
  4. Separate protocol-layer conclusion from suspected model family conclusion.
  5. Run repeated stability tests on the best working route.
  6. Run strict formatting tests.
  7. Run deeper advanced dimensions when the user prioritizes realism over speed.
  8. Inspect family fingerprint evidence and produce a confidence-based judgment.

References to load as needed

  • Main checklist: references/forensics-checklist.md
  • Advanced dimensions: references/advanced-dimensions.md
  • Error/stream/variance: references/error-stream-variance.md
  • Protocol specifics: references/protocol-openai.md, references/protocol-anthropic.md, references/protocol-gemini.md, references/protocol-glm.md
  • Family fingerprints: references/fingerprint-*.md
  • Native deep tests: references/deep-claude.md, references/deep-gemini.md

Final labels

  • high-confidence-focused-or-genuine-route
  • medium-confidence-likely-routed-or-wrapped
  • high-confidence-multi-model-aggregation-pool
  • low-confidence-or-unusable

Use high-confidence-focused-or-genuine-route sparingly. It should require:

  • stable repeated success
  • no strong mixed-pool signal
  • coherent family fingerprint
  • and no obvious gateway-normalization red flags in deep tests

Agent output contract

Return sections in this order:

  1. Declared facts
  2. Availability status
  3. Protocol-layer findings
  4. Suspected model-family findings
  5. Stability findings
  6. Capability/format findings
  7. Advanced-dimension findings
  8. Final judgment
  9. Need-human-review items
  10. Recommended operational posture

Preferred execution

python3 scripts/llm_provider_forensics.py --config /root/.openclaw/openclaw.json --providers omgteam ypemc vpsai --model gpt-5.4 --deep