Spider — Web Scraping Tool

This is the default web scraping method, replacing older approaches like web_fetch.

Trigger Conditions

Use this skill when user says:

Usage Examples

Operation Flow

1. Check Chrome Status

{ action: "status" }

If not running, start it:

{ action: "start" }

2. Open Target Page

{ action: "open", targetUrl: "https://stockpage.10jqka.com.cn/300620/news/", target: "host" }

3. Get Page Snapshot

{ action: "snapshot", targetId: "xxx", maxChars: 20000 }

4. Page Interaction (click, type, etc.)

{ action: "act", targetId: "xxx", request: {"kind": "click", ref: "e33"} }

5. Cleanup: Return to about:blank

{ action: "navigate", targetId: "xxx", url: "about:blank" }

Common Website Templates

Tonghuashun Stock News

URL: https://stockpage.10jqka.com.cn/{stock_code}/news/
Example: https://stockpage.10jqka.com.cn/300620/news/

East Money Guba (Stock Forum)

URL: https://guba.eastmoney.com/list,{stock_code}.html
Example: https://guba.eastmoney.com/list,300620.html

Xueqiu (Snowball)

URL: https://xueqiu.com/S/SZ{stock_code}
Example: https://xueqiu.com/S/SZ300620

Baidu News Search

URL: https://www.baidu.com/s?wd={keyword}&tn=news

Chrome Setup (One-time)

1. Open Chrome Flags:
   • chrome://flags/#enable-experimental-web-platform-features → Enabled
   • chrome://flags/#enable-webmcp-testing → Enabled
2. Fully quit Chrome (Cmd+Q) and restart

Important Rules

1. Use target="host" instead of "sandbox"
2. Must cleanup after each task:
   • If multiple tabs exist, keep only one, close others
   • The remaining tab must navigate to about:blank
   • If multiple about:blank tabs exist, keep only the latest one, close others
   • Use browser action: tabs to check current tab status
   • After cleanup, ensure only one about:blank tab remains
3. Reuse existing tabs, avoid opening new tabs frequently
4. Handle anti-scraping sites: Tonghuashun, East Money need complete JavaScript loading

Error Handling

Default Scraping Priority

1. Spider (Chrome + WebMCP) ← Primary method

   • Suitable for: Finance websites, stock news, forums
   • Advantages: Full JavaScript rendering, interactive

2. web_fetch ← Backup method

   • Suitable for: Simple static pages
   • Disadvantage: Cannot handle JavaScript-rendered pages