360 Secure SkillHub
360 安全技能中心

Llm Provider Forensics

核心功能：通过协议兼容、家族指纹、稳定性等多维度检测，识别LLM端点/模型ID的真实归属，覆盖GPT、Claude等主流LLM家族及混合网关。典型场景：验证模型真伪、区分路由/代理/聚合池、审计可疑网关、对比同模型提供商。输出带置信度的结构化报告，辅助决策。

andyrenxu7255
andyrenxu7255
热度 48.82
最近更新 2026-05-17 08:11:50
分类：开发依赖：代码执行 / 第三方 API / 读写文件应用场景：API 集成 / 调试排错 / 报告撰写

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add llm-provider-forensics
Metadata
name
llm-provider-forensics
description
Forensically verify what model family or routing layer may actually sit behind a claimed LLM endpoint or model ID. Use when an agent must investigate whether a provider is genuine, proxied, aliased, aggregated, wrapped, or currently unusable across OpenAI-compatible protocol layers, GPT/OpenAI, Anthropic/Claude, Google Gemini, GLM/Zhipu, Qwen/Tongyi, Kimi/Moonshot, MiniMax, DeepSeek, and mixed compatibility gateways. Supports deeper family-fingerprint analysis, long-context tests, structured-output stress, refusal and variance profiling, streaming/error clues, repeated stability checks, and cross-provider comparison reports.

LLM Provider Forensics

Agent-facing forensic skill for identifying what an LLM endpoint most likely is.

Trigger conditions

Use this skill when asked to:

  • verify whether a claimed model is genuine
  • identify which family an endpoint most resembles
  • distinguish focused route vs wrapped route vs aggregation pool
  • compare multiple providers claiming to expose the same model
  • evaluate primary/fallback/avoid decisions
  • deeply audit suspicious gateways for GPT / Claude / Gemini / GLM / Qwen / Kimi / MiniMax / DeepSeek behavior

Core rule

Do not output false certainty. Produce a confidence-based operational judgment.

Coverage

Families:

  • OpenAI-compatible protocol layer
  • GPT / OpenAI-style
  • Claude / Anthropic-style
  • Gemini / Google-style
  • GLM / Zhipu-style
  • Qwen / Tongyi-style
  • Kimi / Moonshot-style
  • MiniMax-style
  • DeepSeek-style
  • mixed aggregation pool / compatibility gateway

Dimensions:

  • catalog topology
  • protocol compatibility
  • response schema shape
  • repeated stability
  • strict formatting control
  • family fingerprinting
  • long-context retention
  • structured-output stress
  • refusal/safety style
  • randomness / variance profile
  • streaming / error fingerprints
  • cross-protocol consistency

Current implementation note:

  • openai-compatible now means protocol layer only, not GPT-family proof.
  • The deepest automatic suite is strongest for OpenAI-compatible / mixed gateway providers.
  • Anthropic-native and Gemini-native routes currently have solid protocol/family checks, plus native deep tests, but protocol success alone must not be read as family proof.
  • Treat all family conclusions as confidence-based and inspect references before overclaiming.

Investigation workflow

  1. Identify likely protocol family or families.
  2. Probe catalog/list endpoints when available.
  3. Probe minimal inference endpoints for each plausible protocol family.
  4. Separate protocol-layer conclusion from suspected model family conclusion.
  5. Run repeated stability tests on the best working route.
  6. Run strict formatting tests.
  7. Run deeper advanced dimensions when the user prioritizes realism over speed.
  8. Inspect family fingerprint evidence and produce a confidence-based judgment.

References to load as needed

  • Main checklist: references/forensics-checklist.md
  • Advanced dimensions: references/advanced-dimensions.md
  • Error/stream/variance: references/error-stream-variance.md
  • Protocol specifics: references/protocol-openai.md, references/protocol-anthropic.md, references/protocol-gemini.md, references/protocol-glm.md
  • Family fingerprints: references/fingerprint-*.md
  • Native deep tests: references/deep-claude.md, references/deep-gemini.md

Final labels

  • high-confidence-focused-or-genuine-route
  • medium-confidence-likely-routed-or-wrapped
  • high-confidence-multi-model-aggregation-pool
  • low-confidence-or-unusable

Use high-confidence-focused-or-genuine-route sparingly. It should require:

  • stable repeated success
  • no strong mixed-pool signal
  • coherent family fingerprint
  • and no obvious gateway-normalization red flags in deep tests

Agent output contract

Return sections in this order:

  1. Declared facts
  2. Availability status
  3. Protocol-layer findings
  4. Suspected model-family findings
  5. Stability findings
  6. Capability/format findings
  7. Advanced-dimension findings
  8. Final judgment
  9. Need-human-review items
  10. Recommended operational posture

Preferred execution

python3 scripts/llm_provider_forensics.py --config /root/.openclaw/openclaw.json --providers omgteam ypemc vpsai --model gpt-5.4 --deep