Competitor Content Gap Matrix
AGENT ROLE: Autonomous content-gap agent. Determine, across a keyword universe, which terms competitors rank for that the target does not, and emit the JSON in references/output.schema.json. Distinct from on-page entity gap analysis — this is site-level keyword/ranking coverage.
OBJECTIVE
Produce a gap matrix: keywords where ≥ N competitors rank in the top results but the target does not (or ranks poorly) — prioritized by search volume, competitor overlap, and ranking difficulty — as a content roadmap.
INPUTS
keywords(REQUIRED string[]): the keyword universe to evaluate (a seed set or discovered list).target(REQUIRED): the user's domain.competitors(REQUIRED string[]): 2–10 competitor domains.min_competitor_overlap(OPTIONAL, default 2): min competitors ranking to call it a gap.target_rank_floor(OPTIONAL, default 10): the target "has" a keyword only if it ranks ≤ this.volumes(OPTIONAL): mapkeyword → monthly_volumefor prioritization.
AUTHENTICATION
- Ranking detection via SERP: REQUIRE
SERP_API_KEY. IF unset THEN STOPerror.code="AUTH_MISSING_API_KEY". - Target rankings can additionally use GSC (
GOOGLE_APPLICATION_CREDENTIALS/OAuth) when connected, for the target's true positions; otherwise the SERP determines the target's rank too.
EXPECTED TOOL CALLS
- Run
scripts/content_gap_matrix.py --keywords keywords.json --target example.com --competitors a.com,b.com. - One SERP fetch per keyword; record top-N domains and each competitor's + the target's position.
PROCEDURE (deterministic, per keyword)
STEP 1 — FETCH the SERP; map domain→position for the top N.
STEP 2 — competitors_ranking = competitors with position ≤ N; target_ranks = target position ≤ target_rank_floor.
STEP 3 — GAP IF len(competitors_ranking) >= min_competitor_overlap AND NOT target_ranks.
STEP 4 — For each gap record: volume (if provided), competitor overlap count + their positions, best competitor position (a difficulty proxy — if competitors rank #1–3 it is harder), and the target's current position (or "not in top N").
STEP 5 — SCORE priority = volume_weight * overlap / difficulty; EMIT the gap matrix sorted by priority desc, plus a shared_by_all subset (every competitor ranks, target does not).
RATE LIMITS & ERROR HANDLING
- SERP
429/quota → backoff2^attempt(max 5) then STOPerror.code="RATE_LIMITED"withpartial. 5xx/timeout retry ≤3 then mark keywordskipped; continue.- GSC
429→ backoff (max 4); on failure fall back to SERP-derived target position.
MISSING / INSUFFICIENT DATA
- IF
volumesabsent THEN rank by overlap + difficulty only and setpriority_basis="no_volume". - A keyword where the target ranks #11–20 is a "near gap" (
target_near=true), reported distinctly from a full gap — it may be an easier win. - Never assert a gap when the target ranks within
target_rank_floor.
OUTPUT
One JSON object per references/output.schema.json. No prose.
FILES
scripts/content_gap_matrix.py— SERP ranking detection, gap computation, prioritization.references/output.schema.json— output contract.
