llmwiki

A personal knowledge base that an LLM compiles, not just stores. Raw documents go in, an LLM writes trilingual (EN / 中文 / 日本語) wiki articles with [[wiki-links]], backlinks, and an emergent taxonomy. The MCP server dispatches every tool through llmwiki/operations.py; the CLI exposes the same registry via llmbase ops call; individual HTTP/CLI wrappers are being migrated onto the registry over time.

• PyPI: pip install llmwiki
• CLI command: llmbase (the package name and the command differ)
• GitHub: https://github.com/Hosuke/llmbase
• Demo: https://huazangge-production.up.railway.app

Setup

pip install llmwiki

mkdir my-kb && cd my-kb

cat > .env << 'EOF'
LLMBASE_API_KEY=sk-your-key
LLMBASE_BASE_URL=https://your-endpoint/v1
LLMBASE_MODEL=your-model
# Optional: LLMBASE_FALLBACK_MODELS=backup-1,backup-2
EOF

cat > config.yaml << 'EOF'
llm:
  max_tokens: 16384
paths:
  raw: "./raw"
  wiki: "./wiki"
EOF

Commands

MCP Integration (for AI clients)

{
  "mcpServers": {
    "llmwiki": {
      "command": "python",
      "args": ["-m", "llmwiki", "--base-dir", "/path/to/my-kb"]
    }
  }
}

Tools exposed by the MCP server:

All tools are declared in llmwiki/operations.py — downstream projects register custom ops via operations.register(...) and they become available on CLI + MCP automatically.

Agents mounted on this server can answer from compiled concepts, fall back to raw sources with kb_search_raw when compile glossed a detail, ingest new material mid-session, and trigger healing.

Workflows

Build a KB from scratch

llmbase ingest url https://example.com/topic
llmbase ingest pdf ./paper.pdf
llmbase compile new
llmbase query "What are the key concepts?"
llmbase lint heal

Autonomous mode (deploy once, server keeps learning)

# config.yaml
worker:
  enabled: true
  learn_source: cbeta         # built-in: cbeta | wikisource | both; custom via register_learn_source()
  learn_interval_hours: 6
  compile_interval_hours: 1
  health_check_interval_hours: 24

health:
  auto_fix_broken_links: true
  max_stubs_per_run: 10

The worker starts under the production WSGI entrypoint (wsgi.py → start_worker_thread). Deploy with gunicorn wsgi:app; llmbase web alone does not self-start the worker.

Daily use as agent memory

1. Agent receives a task → calls kb_search for relevant concepts
2. If the compiled answer is too abstract → calls kb_search_raw for verbatim detail
3. Learns something new → calls kb_ingest with the URL
4. Optionally kb_compile to fold it into concepts for next session
5. Periodically kb_lint heals the graph

Key Concepts

• Synthesis, not archiving — LLM reads raw material and writes composed articles; storage is the cheap part
• Two-layer recall — kb_search (concepts) + kb_search_raw (verbatim raw sources)
• Trilingual default — every article has EN / 中文 / 日本語 sections
• 叠加进化 — new data merges into existing concepts, never overwrites
• Domain-agnostic — taxonomy emerges per-domain, nothing hardcoded
• Self-healing — 7-step auto-fix pipeline repairs drift
• Alias resolution — [[参禅]] → can-chan.md across scripts and simplified/traditional
• Registry-backed ops — MCP dispatches every tool through operations.py; CLI exposes the same registry via llmbase ops list / llmbase ops call; direct HTTP/CLI wrappers are being migrated onto the registry

Tips

• --file-back saves Q&A answers into the wiki so future queries benefit
• --tone wenyan for Chinese users (classical Chinese responses)
• Run llmbase lint heal after large ingestion batches
• Web UI /health has buttons for every repair op
• Knowledge graph at /graph — density slider for large KBs
• Timeline at /explore — requires entities: { enabled: true } in config

Security & Privacy

• All data stays local — wiki files are plain markdown on your filesystem
• LLM API key — user-supplied, loaded from .env
• Network access — user-initiated (URL ingest, SSRF-protected) plus corpus plugins (cbeta-learn, wikisource-learn, ctext-book) and the autonomous worker when enabled
• Web server — optional; binds 0.0.0.0 so LAN-accessible by default — front with a reverse proxy or bind override for public exposure
• API secret — cloud deployments (with PORT env) gate most mutating endpoints behind LLMBASE_API_SECRET (auto-generated if unset). Note: /api/ask is open by default and writes Q&A back via file_back; only promotion to concepts requires the secret
• Autonomous worker — opt-in via config, disabled by default
• No telemetry — nothing is sent anywhere except the configured LLM API