Clinical Trial Literature Search / 临床试验文献检索专家

Language

• English guide → README.md · 中文指南 → README_zh-CN.md
• Bilingual auto-switch: the answer language follows the user's question language (English question → English answer, Chinese question → Chinese answer).

Purpose

Retrieve published scholarly literature (peer-reviewed papers, systematic reviews, conference abstracts, preprints) about a drug / disease / method, normalize heterogeneous records from multiple public bibliographic sources into one de-duplicated evidence base, and surface the evidence landscape plus a CSM (cumulative safety monitoring) qualitative subset. Supports trial-planning background, protocol / CSR introductions, and published-safety literature checks.

Positioning within the ct- library

The four A-tier public-intel skills (non-confidential input, network=public-retrieval) are complementary:

Boundaries: ct-registry never fetches paper full-text/abstracts; ct-literature never fetches registry structured metadata. ct-literature --safety surfaces published safety literature — qualitative, must NOT replace ct-safety's FAERS disproportionality. Not sure which skill? Route via ct-advisor; full competitive-intel picture → ct-pipeline directly (§15).

Data Sources

All are public bibliographic APIs — no WAF. The default data sources are OpenAlex (primary) + Europe PMC (on by default) + bioRxiv/medRxiv (on by default); everything else (arXiv, Cochrane, PROSPERO, Semantic Scholar) is opt-in. OpenAlex keyless = 100 credits/day since 2026-02-13; a free key lifts to 100k/day (--openalex-key / env OPENALEX_API_KEY / skill .env auto-load; key never printed). Semantic Scholar is an explicit opt-in source (--with-semantic-scholar + key) and is not considered in the default pipeline.

PROSPERO is a reserved source (2026-08-12): its public REST auth header is undocumented; unauthenticated probes return {"status":"error",...}. --with-prospero is a dormant interface: without a token it degrades to a graceful no-op skip (returns None, no file written) and is not claimed functional. Supply --prospero-token (+ --prospero-header) to exercise it; parser is schema-tolerant (JSON + XML) but must be re-validated against a real 200 before declared done. No token application planned.

Clinical guideline sources (--with-guidelines, opt-in · LOCAL corpus)

Guidelines are version-pinned reference standards — at analysis time we read a pre-built LOCAL corpus, never "fetch latest" per run.

Corpus boundary (audit note 2026-09-07): every entry is a manually curated, version-pinned pointer to an organisation-issued document, but the builder's 12+ sources also surface reviews / consensus / adherence analyses and AI-in-medicine commentary. Such items are kept as background pointers only and are not authoritative clinical guidance — always resolve the linked source before relying on it; a retrieved:false pointer is an honest placeholder, never a fabricated citation.

🔒 Data-protection split. The skill tree ships pointer-only (references/guidelines/guidelines_index.json: org/title/URL/version — publish-safe). Full-text documents are NEVER written into the skill — they live in the author's self-controlled Coze KB (or an EXTERNAL local cache ~/.workbuddy/ct-guideline-docs, opt-in via --download, off by default); ct-advisor consults that KB for native guideline Q&A.

• Author / build-time (network): python adapters/build_guidelines.py --topic <topic> --run aggregates 12+ sources → guidelines_index.json (96 curated entries, schema v1). SAFE PREVIEW: omit --run (dry-run, no network/write).
• Analysis-time (zero network): --with-guidelines on the main pipeline → adapters/guideline_corpus.load() reads the local index → guidelines.json + guidelines block in .merged.json.

Build-time portal fetch (adapters/portal_fetch.py): every fetcher is wrapped so it never raises — failed fetch degrades to the honest pointer, so the corpus is always honest (build once, read many). Each record carries access (api/portal) + retrieved; guideline_corpus.load() filters by topic/org and returns corpus_missing (with the builder command) if the index is absent. ¹ NICE REST auth undocumented (like PROSPERO) — skip until a token works. ² TRIP requires a commercial key.

Features

Unified work schema

{ source, id, title, authors, year, publication_date, publication, journal_iso,
  type, study_type, cited_by_count, url, open_access_url,
  pmid, pmcid, doi, abstract_snippet, mesh, concepts, keywords, funders,
  language, is_retracted, is_safety, is_preprint, volume, issue, page,
  affiliations, sources }

Output

Standard deliverables = HTML + Excel only. Present lit_report.html to the user as the result, and remind them of the optional add-ons below (the report itself shows a "还能做什么 / Options" strip with these hints).

• Per-source payloads: openalex.json / europepmc.json / semantic_scholar.json / biorxiv.json / medrxiv.json / arxiv.json (enabled only)
• lit_report.xlsx — Excel delivery (excel_style; --no-xlsx to skip): Overview → Literature master → Safety-related, KPI cards, charts, is_safety amber highlighting. The complete result — user can keep filtering / pivoting on top of it.
• lit_report.html — self-contained HTML report (inline CSS, offline; --no-html to skip); inline-SVG PRISMA funnel when --prisma. Default deliverable to open.
• On-demand citation formats (default OFF — ask the user or use --export-bib): references.bib / references.ris / references_<style>.md (Zotero RIS / BibTeX / APA etc.). OA-PDF downloads are likewise on request: --download-pdf attempts ALL records that carry an OA URL or DOI (batched, legal OA sources only); users may instead ask for specific DOI / PMID(s) or the top-N works. Server-side supplement chain: when the A-path direct links all fail, the coze endpoint runs a supplement chain — Unpaywall best-OA → Europe PMC PMC-OA → PPR preprints (bioRxiv/medRxiv, author-verified) — before the (off) browser fallback; the local skill only calls, it no longer implements multi-channel download algorithms.
• obsidian/ (--obsidian) — per-paper notes + Literature MOC.md; zotero.csv / zotero.ris (--zotero)
• .merged.json gains additive prisma + per-work relevance_score / prisma_included blocks

See references/sop.md for the full command catalogue.

Requirements

• Python 3.10+ (Anaconda C:\Tools\anaconda3\python.exe recommended).
• requests optional (fetch scripts use stdlib urllib); matplotlib optional (future trend charts).
• Network: read-only public bibliographic APIs.

⚠️ Safety

• Default SAFE PREVIEW: scripts only generate / display; network requests run only with explicit --run.
• Reads public publications ONLY, zero confidential research / subject data input (A-tier — non-confidential input, per §11; API keys are local config, never research data).
• --safety literature is qualitative — never feed it into FAERS disproportionality; it only corroborates ct-safety qualitatively.
• Output is for reference / background only, not a regulatory submission.

Implementation

# Primary: OpenAlex only (no key)
python scripts/ct_literature.py --topic "osimertinib" --review-type systematic-review --year-from 2018 --safety --run --out-dir ./out

# Add Europe PMC (default ON) + Semantic Scholar (may 429 -> skipped)
python scripts/ct_literature.py --topic "osimertinib" --with-europepmc --with-semantic-scholar --run --out-dir ./out

# Cochrane-only retrieval (focus on the Cochrane Database of Systematic Reviews)
python scripts/ct_literature.py --topic "NSCLC" --cochrane --with-europepmc --run --out-dir ./out

# Clinical guidelines: build once (network), read many (zero network)
python adapters/build_guidelines.py --topic "diabetes" --run          # author/build-time; omit --run = SAFE PREVIEW
python scripts/ct_literature.py --topic "2型糖尿病" --with-guidelines --run --out-dir ./out

OpenAlex API key (recommended since 2026-02-13)

Keyless is capped at 100 credits/day; a free key lifts to 100k/day. Zero-friction: drop the key into the skill's .env (copy from .env.example) — no extra flag needed. http_utils.load_openalex_key() auto-resolves: env OPENALEX_API_KEY → skill-root .env → scripts/.env (key value never printed). Explicit provision also works (--openalex-key). Application steps, quota, troubleshooting → references/openalex_key.md.

Errors

See references/errors.md for the full error catalogue (network / 429 / 401 / empty results / DOI dedupe).

Pipeline

• ct-registry → ct-literature: landscape hypothesis seeds the literature search topic.
• ct-literature → ct-pipeline (intel evidence dimension), → ct-protocol / ct-csr (background), --safety → ct-safety (qualitative corroboration).

Cross-Database Search Mode

A cross-database planning layer (Embase / Cochrane / Web of Science + preprint Tier P, adapted from multi-database-literature-collector, AIPOCH MIT) builds search strategy; live fetch still runs the six sources. See references/multi-db-search.md. The Cochrane leg is now directly automatable via --cochrane (a verified Europe PMC journal filter — identical to meta-analysis's in-skill dedup probe), so CDSR needs no manual browser step.

Natural-Language Dialogue — Triage-First (mandatory)

NL dialogue MUST triage before choosing the interaction shape. Mandatory per §6.2 + references/search_menu.md — this step is never skipped.

Step 0 Triage (four buckets, run on the user's first message)

Red lines

• Never skip triage and run --run directly. Even with a clear topic, missing key parameters (review_type / year / sources / …) makes the Complex menu flow mandatory.
• Never decide for the user. The Complex menu must carry a "still unsure → explain the differences" entry (option ③ in the CN UI: 说「详细解释这些选择之间的差异」，我先讲清临床与统计含义再让你决定).
• The §12 keyword-system confirmation gate is mandatory: whenever the user provides parseable search terms, build the Manifest and let the user review the expanded keywords before running.

Cross-turn continuity (continuity.md mode A, high risk)

ct-literature's query spec lives ONLY in the conversation thread (OpenAlex / Europe PMC etc. are stateless remotes); a dropped field silently searches the wrong scope.

Echo block (append after every run)

## 当前检索设定：topic=… | type=… | year=… | safety=… | sources=… | max=… | verify=…

• Fixed prefix ## 当前检索设定： (inherited by search_menu.md §13)
• Fields are key=value, pipe-separated; missing defaults use —; never omit fields
• Full field list: topic / type / year / safety / sources / max / verify

Follow-ups change only the changed fields

The LLM must read the MOST RECENT settings block in the conversation, override only the fields the user changed this turn, keep the rest as-is, then proceed to the §12 gate or run.

merge_spec fallback (default path)

ct-literature is a high-risk skill (continuity.md §5.1); merge_spec.py is the DEFAULT path on follow-ups:

echo '{"prev":{previous full spec},"cur":{this-turn partial}}' \
  | python scripts/merge_spec.py

Feed the merged output into the §12 keyword gate. prev comes from the thread's latest settings block — never persisted, never cached across turns.

Local-only continuity

OpenAlex / Europe PMC / bioRxiv / medRxiv are all stateless remotes — they do not remember your previous search parameters. Continuity (parameter inheritance) MUST be solved locally (thread-resident + echo block + merge_spec); there is no "the remote will remember" fallback.

Execution flow

user request → Step 0 Triage
  ├─ Simple → §4.2 preview confirm → §12 keyword gate → run → §13 echo
  ├─ Complex → §4.1 initial menu → parameters complete → §12 keyword gate → §4.2 preview confirm → run → §13 echo
  ├─ Vague → grill-me (≤2 rounds) → re-triage
  └─ Middle → answer directly (no menu)

Bug Reporting (§20.3, adapter: adapters/bug_report.py)

• Trigger: (A) explicit user request ("report a bug" / "反馈问题" / "提交错误报告") → straight to two-stage confirmation, unlimited per session; (B) strong signal (unexpected non-zero exit / engine or compute error / user explicitly questions the result) and the same operation was retried ≥1 → at most 1 unsolicited proposal/session.
• Two-stage confirmation (2026-08-21): ① propose-with-preview — bilingual confirm_prompt together with the full sanitized report (invite a problem description; re-render before consent) → ② on explicit consent, send_to_endpoint (auto action=report, endpoint https://ct-bugreport.coze.site/run, token = §5 public credential). Decline → never re-propose this session.
• Sanitization is hard: 11-key whitelist only (skill / version / error_type / error_code / engine_status / description / locale / query_origin / session_hash / attempts / test) — never raw data or subject records; description is the single user-reviewed free-text field. No cloud call → save_local_report() (local md + author email).
• Client-only: sends report only; governance actions belong to ct-update (author side). Post-send (2026-08-22): endpoint returns history → reply via confirm_thanks + build_followup (bilingual, locale-switched).

Invoke: python adapters/bug_report.py --error-type <t> --description "<free text>" [--send] (add --send only after the user confirms).