360 Secure SkillHub
360 安全技能中心

HotTrender Basic Crawler

核心功能为轻量化热点爬虫，支持四地区每日热点、垂类及自定义关键词热点抓取，优先使用捆绑的爬虫运行时与现有脚本，不添加推送、存储等额外功能。典型场景包括获取指定地区每日热点、特定垂类/关键词热点，排查抓取有效性问题。输入为相关指令，通过指定脚本执行，输出热点数据文件，修改代码前需先检查现有能力是否满足需求。

7487
7487
热度 49.91
最近更新 2026-05-11 17:44:03
分类：开发依赖：代码执行 / 读写文件 / 联网浏览搜索应用场景：数据处理 / 自动化脚本 / 研究

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add hottrender-dingtalk-bot
Metadata
name
hottrender-dingtalk-bot
description
Use when users need a lightweight HotTrender crawler for four-region daily hotspot trends or custom keyword/vertical hotspot discovery. Prefer the bundled basic crawler runtime and existing provider scripts before writing code. This skill intentionally excludes DingTalk push, OSS publishing, ActionCard pages, lp-ads workspace, worker queues, databases, and LLM summaries.

HotTrender Basic Crawler

Core Rule

For four-region daily hotspot trends or vertical/custom-keyword hotspot discovery, use the bundled crawler runtime first. Do not reimplement platform crawling until the existing providers and scripts are checked.

Before changing code, answer these questions:

  1. Is there already a script, API, provider, doc, or test covering this need?
  2. Can the user goal be satisfied by running or configuring that capability?
  3. If not, what exact gap remains, and where is the smallest extension point?

Only edit code after that evaluation.

Repository Layout

This skill bundles a sanitized basic crawler runtime under assets/hottrender-runtime/. It does not bundle DingTalk, OSS, ActionCard, lp-ads, worker queues, databases, logs, LLM, or any secrets.

First resolve the runtime path from environment variables, the current workspace, or the bundled runtime:

HOTTRENDER_APP_DIR   # directory containing scripts/fetch_daily_trends.py

If HOTTRENDER_APP_DIR is missing, install the bundled runtime:

python assets/install_hottrender_runtime.py --target ./HotTrenderRuntime
export HOTTRENDER_APP_DIR="$PWD/HotTrenderRuntime"

If variables are missing but a local checkout may exist, discover it safely:

find "$PWD" "$HOME" -maxdepth 5 -path '*/scripts/fetch_daily_trends.py' 2>/dev/null

Fast Path

Use these references only when needed:

Operating Workflow

  1. For "四地区热点", "每日热点", "daily trends", or "jp/us/tw/kr", start from scripts/fetch_daily_trends.py.
  2. For "垂类热点", "关键词热点", "自定义关键词", or "custom keyword", start from scripts/fetch_keyword_hotspots.py.
  3. For "抓取是否有效", "平台数据不对", or "为什么没结果", inspect configs/providers.yaml, run offline mode first, then real mode.
  4. For code changes, keep the runtime basic. Do not add DingTalk, OSS, lp-ads, database, worker, or LLM features back into this skill.

Guardrails

  • Never print API keys, cookies, tokens, msToken, proxy credentials, or other secrets.
  • Do not fabricate live platform data. Offline/sample mode must be called out as offline/sample.
  • Do not introduce push, publishing, database, queue, or workspace features into this basic crawler.
  • If the user has no HotTrender checkout, use the bundled runtime installer before proposing code rewrites.
  • Keep changes scoped: provider logic in src/providers, orchestration in src/crawler.py, CLI entrypoints in scripts/.

Verification

Prefer focused verification:

cd "$HOTTRENDER_APP_DIR"
python -m pytest tests/test_basic_crawler.py -q
python scripts/fetch_daily_trends.py --config configs/providers.yaml --output out/daily_trends.md