Content Engine

Cross-platform content deconstruction + knowledge graph. The bottom layer is a graph that grows over time; the top layer hosts multiple modes across multiple platforms.

中文版 / Chinese: see ~/.agents/skills/content-engine/

Mode Roadmap

Platform Roadmap

Current state: this document fully describes the Xiaohongshu v1 implementation. Future platforms will reuse the same architecture (extract_{platform}.py + content_engine/{platform}/ submodules), and the graph/ knowledge graph is shared cross-platform.

This document is the v1 execution manual — covering Xiaohongshu + deconstruct mode only.

Platform compatibility: This skill runs in OpenClaw (as a personal agent skill) and Claude Code. Scripts use Python 3.10+ stdlib (no external deps); the only system command needed is ffmpeg. File I/O assumes your agent has Read / Write tools (in OpenClaw these map to apply_patch / Exec / Web browser).

Architecture: graph/ is the brain

content-engine-en/
├── SKILL.md              ← what you're reading; the agent's entry point
├── graph/                ← knowledge graph (shared "memory / soul / context" across modes)
│   ├── index.md              brand briefing, agent reads first
│   ├── brand/{brand-voice,brand-story}.md
│   ├── platforms/xiaohongshu.md     XHS playbook (only platform in v1)
│   │                                 (v1.1+ will add douyin.md / wechat-channels.md / ...)
│   ├── audience/segments.md          audience segmentation
│   └── engine/{hooks,style-tags,taboo}.md
├── references/{output-template,example-video,example-image}.md
└── scripts/                          ← symlink to ../content-engine/scripts/ (shared code)
    ├── extract_xhs.py                XHS one-shot orchestrator: link → workspace
    │                                  (v1.1+ will add extract_douyin.py etc)
    └── content_engine/               Python package (zero deps)
        ├── client.py                  TikhubClient
        ├── parsers.py                 NoteData / Comment dataclass parsing
        ├── linkresolve.py             short link → note_id
        ├── video.py                   download + ffmpeg frame extraction
        ├── images.py                  image post downloader
        └── models.py                  dataclass definitions

Two ironclad rules:

1. graph/ files can be empty templates — deconstruction still runs, just falls back to "objective deconstruction" mode
2. New hooks / style words discovered during deconstruction are auto-written back to graph/engine/. The graph grows.

When to trigger

• User gives an XHS link and says "deconstruct this" / "study this" / "why did this go viral?"
• Competitive analysis during content planning
• The "research first" step before generating new content

Input

Output

Markdown file → docs/deconstructions/{id}-{slug}.md.

Full field definitions in references/output-template.md. Examples in references/example-video.md / example-image.md.

Workflow

Step 0: Detect graph state → choose mode

Use your agent's native tools directly (avoid bash globstar / realpath compat issues):

1. Locate the skill root (where SKILL.md lives)
2. Use Read or Grep tools to scan graph/**/*.md for # TODO: markers

Simplest: a single Grep call:

Grep pattern: "^# TODO:"  path: <skill_root>/graph/  output_mode: files_with_matches

Mode A vs B differences in the "Takeaways" field: see the dual-version comparison at the end of references/example-video.md.

Wikilink convention [[brand/brand-voice]] between graph files: this is Obsidian-style, pointing to graph/brand/brand-voice.md (no .md suffix). When you see [[X]], Read the corresponding file to load context.

Steps 1-3: One-shot data fetch → workspace

One command does it all: link resolution / metadata fetch / comments fetch / video download + frame extraction / image download for image posts.

⚠️ v1 supports Xiaohongshu only. Douyin / WeChat Channels / Bilibili etc. are on the roadmap (v1.1+) with corresponding extract_douyin.py / extract_wechat_channels.py scripts.

python3 scripts/extract_xhs.py "<XHS link / note_id / share text>"
# Default workspace: {tempdir}/content-engine/{note_id}/
# Custom: --out /your/path

First-run environment check:

python3 scripts/extract_xhs.py --check

(Checks Python version / ffmpeg / TIKHUB_API_TOKEN / network / workspace writable. See "Setup" section below.)

Workspace artifacts (default {tempdir}/content-engine/{note_id}/, cross-platform):

Error handling:

• API 401/403 → non-zero exit, tell user and stop
• Comments API failure → comments.json written as {"_error": "..."}; "comment keywords" field becomes "⚠️ Not retrieved"
• Non-XHS link → tell user "v1 only supports Xiaohongshu" and stop

Common flags:

• --no-video skip video download (metadata-only mode)
• --no-comments skip comments
• --fps 1.0 force frame rate (default auto-adapts to duration)

Step 4: Multi-modal deconstruction

Step 4a · Required reading before deconstruction (graph hard gate)

Always Read first:

• graph/platforms/xiaohongshu.md — sections "What to focus on when deconstructing" + "Platform viral formulas" + "Taboos"
• graph/engine/style-tags.md — full style dictionary (Step 5 style tag field uses this)
• graph/engine/hooks.md — full hook library (Step 5 emotion-hook field uses this)

If in brand-aware mode, also Read graph/brand/brand-voice.md + graph/brand/brand-story.md + graph/audience/segments.md.

Step 4b · Video branch (type == "video")

1. Read frames in order (frame_001.png ...). Mental-note for each frame: shot type / subject / action / background / props / camera direction. Don't output N rows of stream-of-consciousness — accumulate material for aggregation in next step.

2. Aggregate into time segments for the "Reference content deconstruction" field. Core rule:

   ✅ Good (aggregated + dense)	❌ Bad (stream of consciousness or empty)
   7-12s ｜ Camera: locked → slow push ｜ Shot: close-up → extreme close-upVisual: emerald-green collar and placket of vest, jade buttons + white beaded geometric embroidery, paired with white jade pendant necklace as styling demo	7s ｜ close-up ｜ collar8s ｜ close-up ｜ collar9s ｜ close-up ｜ button...
   	7-12s ｜ Visual: very pretty clothing detail, exquisite craftsmanship

   Merge rules:

   • 2+ consecutive frames with same subject/shot type → merge into one segment
   • Subject/shot change → start new segment
   • Single-frame holds <2s usually don't get their own segment
   • Use specific nouns (emerald green, beadwork, jade button) not adjective stacking (high-end, exquisite, beautiful)

3. Voiceover/subtitle text:

   • Combine on-screen captions + note.json.desc
   • Pure visual + no captions → write "No voiceover/subtitle, pure visual storytelling" + list bottom-watermark info

4. Voiceover logic analysis: write in layers, each layer with timestamp + one-line function:

   • Example: "Layer 1 · Establish contrast and curiosity (0-12s): the '75-born + 2000m² store' numeric contrast triggers curiosity"
   • Common structures:
     • Hook open → scene immersion → product/USP → identity elevation → CTA
     • Contrast open (number/conflict) → story setup → values → CTA
     • Craft close-up → cultural meaning → emotional resonance → tag elevation

Step 4c · Image branch (type == "normal")

1. Read images in order (images/image_NNN.jpg)
2. Each image: composition / elements / style / role (in the set: cover / detail / outfit / scene)
3. Aggregate into "Reference content deconstruction" by image order: "Image 1 (cover): ... / Image 2: ..."

Step 5: Extract remaining fields

Step 5a · Field-fill table (graph influence)

Step 5b · Quality bar for subjective fields

See "Field definitions + Anti-Pattern" section in references/output-template.md. Core principles:

• Viral theme explains "why it went viral" (mechanism), not "what it is" (description)
• Emotion hook writes "what technique elicits what emotion" (two-layer), not a single isolated word
• Style vs Scene vs Emotion-hook: style = "how it looks/feels", scene = "where it happens", emotion-hook = "what it stirs in the user's mind" — never confuse the three

Step 5c · Comment keyword semantic classification (you do it, no regex)

Why no regex: language has infinite variations ("how do I buy" / "what's the price" / "is it pricey" / "how much"), regex always misses; regex also can't handle semantics ("price isn't a problem" isn't an inquiry; "isn't this silk?" isn't an objection). You (agent) have full language understanding — do this directly, you're 100x better than regex at this.

Data source: comments.json already filtered by parser — is_pinned=True (merchant-pinned / anti-scam) is auto-flagged and skippable; the rest are real user comments.

Four classes (by "what the user is doing"):

Output format (mandatory evidence):

- {keyword label} ({N} raw comments: "text 1" "text 2" "text 3") — {one-line interpretation / conversion signal judgment}

Hard anti-fabrication rules:

1. Each keyword must be backed by 1-3 original comment texts (copy directly from comments.json, no rewriting)
2. Keywords without raw text evidence are not allowed — no fabrication
3. Questions are not objections: "isn't this silk?" is a neutral question (goes to ask); "please don't call this 新中式" is an objection
4. Same comment can fall into multiple classes — "how do I buy this love the green" is both ask and praise; quote it under both
5. comments.json is [] or has _error → write "⚠️ Comment data not retrieved", don't infer from desc

Good vs Bad:

✅ Good (with evidence + interpretation):
- how-do-i-buy (5 raw comments: "how do I buy the green pants" "how to purchase, online?" "how do I buy this love the green") —
  highest-frequency conversion signal
- how-much / pricing (2 raw comments: "how do you sell this" "what's the price for this set") —
  another way of asking pricing
- objection-traditional-attire (1 raw comment: "this is Manchu attire, please don't call it 新中式" 👍 1) —
  only 1 comment but liked, signals tag-usage edge case

❌ Bad (no evidence / fabricated):
- how-do-i-buy, how-much, need-link (just listing words, no raw text — forbidden)

❌ Bad (misclassifying questions as objections):
- objection (comment: "is this silk?")  ← this is a question, not an objection

Step 6: Write back to graph (system gets smarter)

After deconstruction, proactively review and write back:

Strict writeback location rules:

Writeback principles:

1. Append-only, never overwrite
2. Every entry must include "source = task ID" + "date / data"
3. If conflict with existing graph entries → don't write; emit ⚠️ in output for human resolution
4. Hit existing hook/tag → don't duplicate; just mark "reuses existing graph entry" in deconstruction card

Step 6.5: Pre-output self-check (mandatory checklist)

Every line must ✓; failing one means you don't proceed to Step 7:

□ All 18 Excel fields filled, no skips
□ All 5 metadata items (author/time/engagement/note_id/type) pulled live from API, not fabricated
□ Style / Scene / Emotion-hook are not confused (see output-template.md)
□ Reference body copy is desc original (with emojis + line breaks), not paraphrased or trimmed
□ Each comment keyword backed by raw text from comments.json; if comments.json has `_error`, write "⚠️ Not retrieved"
□ Voiceover logic analysis is written in layers (hook/setup/elevation/CTA), not a single paragraph
□ Style tags hitting graph dictionary are marked "existing"; new ones marked "new"
□ Emotion hook hitting existing graph pattern is explicitly noted; new patterns queued for Step 6 writeback
□ Reference hashtags pulled directly from note.json.hashtags (parser already cleaned [话题])
□ Step 6 writeback: explicitly state "N items" or "none"; each item has source/date
□ Objective mode: append "graph/ not populated" notice at the end

Step 7: Publish deconstruction card

Step 7a · Generate slug

From title, generate filename / doc-name slug:

import re
slug = re.sub(r"[^\w一-龥\-_·]+", "-", title)[:30].strip("-") or "untitled"
# e.g., "Shenzhen 新中式｜what does wearing 江南春色 feel like" → "Shenzhen-..."

Final naming: {id}-{slug} (e.g., AIC-260426-001-Shenzhen-deep-dive).

Step 7b · Output (branches based on agent environment)

Preferred: Feishu (Lark) Docx (when running in OpenClaw with the Lark official plugin)

The OpenClaw Lark plugin gives the agent native tools to create cloud documents. In OpenClaw:

1. Use the Lark plugin's "create cloud doc" tool (exact tool name varies by plugin version), passing the full markdown content
2. Title is {id}-{slug}
3. Get the Feishu doc URL; record it for Step 7c

Fallback: local markdown (Claude Code / no Lark plugin / Lark tool failed)

# Use the Write tool to write to:
docs/deconstructions/{id}-{slug}.md

Decision logic:

• Agent self-check: do I have a "create cloud doc" / Lark-document tool in my current session?
• Yes → publish to Feishu, don't also write locally
• No → write locally directly

Note: This skill does NOT wrap Feishu API. The OpenClaw Lark plugin handles auth / upload / conversion; the agent only needs to call the plugin's tools. Claude Code users who want Feishu publishing must manually copy the markdown into a Feishu doc.

Step 7c · User summary (fixed 4 lines)

1. Subject: {title / author / duration / engagement} — one line
2. Strongest insight: {1 core hook or counter-intuitive finding} ({data evidence, e.g., "save-to-like ratio 70%"})
3. Published to: {Feishu URL or local absolute path}
4. Graph writeback: {N items; list top 3, abbreviate rest; if 0, explicitly state "none"}

Boundaries and principles (re-emphasized)

1. No fabrication: API failure / video download failure / unrecognizable subtitles → mark "Not retrieved"
2. Deconstruction is observation, not commentary: factual fields write "the visual shows X", not "this looks great". Subjective judgment only allowed in three fields: emotion-hook / viral theme / takeaways
3. Graph is append-only: writebacks don't overwrite; conflicts get ⚠️ for human resolution
4. Token control: when video frames > 30, aggregate by time segments first (1 representative per 5s) before detailed description
5. No content generation: this mode only outputs deconstruction cards + graph writeback. Generation is v2.

Setup

System requirements

No pip dependencies — scripts use Python stdlib only.

TikHub API token (required)

TikHub is a paid third-party XHS data API. Sign up → Dashboard → API Keys.

Configure (pick one):

# Option 1 (temporary): environment variable
export TIKHUB_API_TOKEN='your_token'

# Option 2 (recommended for long-term): XDG config dir
mkdir -p ~/.config/content-engine
echo 'TIKHUB_API_TOKEN=your_token' > ~/.config/content-engine/.env

# Option 3: drop a .env in cwd or skill root (see .env.example)

Token lookup order (first found wins):

1. TIKHUB_API_TOKEN env var
2. $CWD/.env
3. ~/.config/content-engine/.env (XDG standard)
4. Skill-root .env

Verify

python3 scripts/extract_xhs.py --check

Reports each check ✅/❌/⚠️ with fix instructions.

Mainland China users

• Main domain api.tikhub.io requires a proxy from inside China
• Mirror: api.tikhub.dev (no proxy needed) — set TIKHUB_BASE_URL=https://api.tikhub.dev in .env

Feishu / Lark publishing (OpenClaw users only)

This skill does not bundle Feishu API code. To enable auto-publishing of deconstruction cards to Feishu Docx, install the OpenClaw Lark official plugin:

npx -y @larksuite/openclaw-lark install

Details: OpenClaw Lark official plugin docs

Once installed:

• Agent in OpenClaw gains native "create cloud doc / read cloud doc / update cloud doc" tools
• SKILL.md Step 7 will direct the agent to use those tools for publishing
• Credentials are managed by the plugin; this skill needs zero Feishu config

Claude Code or other environments: deconstruction cards save to local docs/deconstructions/; copy to Feishu manually if needed.

Related / Credits

• analyze-xhs skill: account-level analysis (not single-post)
• Architecture inspiration: Ronin · How To Build Own Content Engine (the Skill Graph idea: use .md + wikilinks as the agent's "memory / soul")
• Chinese version: ~/.agents/skills/content-engine/