omni-scraper

Scrape any webpage via Claw School's proxy. Amazon URLs return structured JSON automatically — no HTML parsing needed.

Setup

You need a CLAW_KEY from claw-school.com. Purchase one and set it in your agent environment:

CLAW_KEY=CLAW-XXXX-XXXX-XXXX-XXXX

Scrape a URL

curl -s -X POST "$CLAW_API_BASE/api/scrape" \
  -H "Content-Type: application/json" \
  -d '{
    "claw_key": "'"$CLAW_KEY"'",
    "url": "URL_HERE",
    "mode": "scraper"
  }'

Mode options:

• scraper — headless browser, handles JS-rendered pages (default, use for Amazon)
• proxy — raw HTTP, faster for static pages and REST endpoints

Optional parameters

Response format

Amazon product page (PDP) — structured fields extracted automatically:

{
  "url": "https://www.amazon.com/dp/B07DN8ZJRL",
  "status": 200,
  "parsed": {
    "asin": "B07DN8ZJRL",
    "title": "Echo Dot (3rd Gen)",
    "price": "$29.99",
    "rating": "4.7 out of 5 stars",
    "review_count": "847,231 ratings",
    "bought_past_month": "10K+ bought in past month",
    "bsr": ["#1 in Smart Speakers"],
    "badges": ["Amazon's Choice"],
    "bullet_points": ["..."],
    "reviews": [{ "author": "...", "rating": "5.0", "title": "...", "body": "..." }]
  },
  "body": null,
  "error": null
}

Amazon search page (SERP) — ranked results with pricing:

{
  "url": "https://www.amazon.com/s?k=iphone+case",
  "status": 200,
  "parsed": {
    "keyword": "iphone case",
    "result_count": 48,
    "results": [
      {
        "position": "1",
        "asin": "B0XXXXX",
        "title": "...",
        "price": "$12.99",
        "rating": "4.7 out of 5 stars",
        "review_count": "23,456 ratings",
        "bought_past_month": "5K+ bought in past month",
        "sponsored": false,
        "ac_badge": "Amazon's Choice"
      }
    ]
  },
  "body": null,
  "error": null
}

Note: If parsed fields are sparse (parse quality low), the response includes body with raw HTML as fallback so you can extract data yourself.

Any other URL — raw HTML in body:

{ "url": "https://example.com", "status": 200, "parsed": null, "body": "<html>...</html>", "error": null }

Error codes

Get a CLAW_KEY

Visit claw-school.com to purchase access. Each key is tied to one agent. Keys do not expire.