Falsifiability
Overview
A meaningful empirical claim must specify what observations would refute it. Claims that resist all possible refutation are not science — they are unfalsifiable belief. Formalized by Karl Popper (1934): science progresses not by accumulating confirmations but by surviving rigorous attempts at falsification. More-specific claims are more falsifiable; ad-hoc modifications that explain away failures destroy a claim's scientific status.
Composes with confirmation-bias (falsifiability is the structural counter), abductive-reasoning (generates hypotheses; this skill tests them), bayesian-reasoning, critical-thinking.
When to Use
- Designing OKRs, KPIs, or strategic goals; writing or evaluating investment theses
- Designing experiments (A/B tests, product hypotheses, market entry)
- Evaluating consultant/advisor recommendations or diagnosing vague leadership claims
- Someone says "what would change your mind," "how would you know you're wrong," "Popper"
- Stress-testing an AI/AGI hype claim ("AGI is near," "the model truly understands," "our AI adoption is working") — demand what evidence would disprove the capability or safety claim
Not when: genuinely non-empirical (philosophical, ethical, aesthetic); test cost exceeds its value.
Coaching Novices (Adaptive Front Door)
- Engine mode: user has a specific claim → run The Process directly.
- Coach mode: user is new → guide step by step.
In Coach mode, respond one step at a time. Each [WAIT] is a hard stop — output only that step's question, then stop.
- One-line: before acting on a claim, ask what observation would refute it — if "nothing would," it's belief, not knowledge.
- Check fit: if the claim is genuinely metaphysical (ethical, aesthetic), falsifiability doesn't apply.
- Elicit the claim and its current evidential basis.
[WAIT — do not advance until user responds]
- Ask: what specific observation would falsify this? When and how would you observe it?
[WAIT — do not advance until user responds]
- Close: falsifiability conditions specified + monitoring plan + commitment to act on disconfirming evidence.
[WAIT — do not advance until user responds]
The Process
Step 1 — State the claim: claim / who asserts it / decision dependent on it / current evidential basis.
Step 2 — Test whether empirical: claim about how the world works (empirical) or values/aesthetics (non-empirical)? If non-empirical, stop here.
Step 3 — Specify falsification conditions: complete "This claim would be falsified if I observed: ___" — specific, observable, time-bounded. If you cannot complete it, the claim is not falsifiable as stated.
Step 4 — Plan to observe: when is the observation possible / how measured / who tracks it / threshold for "falsified."
Step 5 — Pre-commit to action: if the falsifying observation occurs, what will you do? Is any theory modification itself falsifiable?
Step 6 — Iterate: confirmed → keep monitoring; falsified → revise/abandon; unfalsifiable → recognize as belief.
Output Template
# Falsifiability Analysis: <claim>
Claim: | Asserted by: | Decision at stake: | Current basis:
Empirical: Y/N | Observation type:
Falsified if: | Threshold: | Observable when/how: | Owner:
If falsified, action: | Ad-hoc preservation risk:
→ Method in Action: Popper 1934 + Eddington 1919 Eclipse + Modern Applications → 2026 lens: Separating falsifiable from unfalsifiable AI claims (2024–2026)
Pack: Vague → Falsifiable
| Vague (unfalsifiable) | Falsifiable form |
|---|---|
| "We have PMF" | "≥40% of users would be 'very disappointed' without the product" |
| "Our outbound is working" | "5% of cold emails convert to qualified opps within 30 days" |
| "Our culture is strong" | "Employee NPS ≥40 in next quarterly survey" |
| "This stock is undervalued" | "Stock <12 P/E within 12 months; otherwise thesis is wrong" |
| "Users want feature X" | "≥30% complete the new flow within 14 days of launch" |
Applying It Well
- Specify falsifiability conditions before the observation window opens — post-result conditions are rationalization.
- More specific = more falsifiable. Quantify wherever possible.
- When a prediction fails, ask: is the modification of your theory itself falsifiable?
- Apply to others: "What would change your mind?" is the highest-leverage critical-thinking question.
→ Primary sources: references/sources.md
Common Rationalizations
[D] = designed upfront | [O] = observed in real use. [O] entries are more valuable.
| Fake move | Reality |
|---|---|
| [D] "I just have a strong gut feeling" | Gut feelings are not falsifiable. Without a test, you can't tell right from wrong. |
| [D] "We need more data to decide" | If you can't specify what data would change your mind, you're not doing data-driven analysis. |
| [D] "The strategy just needs more time" | Specify the timeframe and metrics in advance. |
| [D] "It's because of external factors" | If external factors can always be invoked, the original claim was unfalsifiable. |
| [D] "It's too early to evaluate" | If you can't specify when evaluation is appropriate, the claim is unfalsifiable. |
| [D] "We're in a special situation" | This defense converts a falsifiable claim into an unfalsifiable one. Resist. |
| → Add [O] entries here after each real use — paste the actual failure pattern | What went wrong and why |
Red Flags
- Claim asserted without specifying what would refute it
- "The strategy is working" without metrics or thresholds
- Predictions modified ad-hoc to accommodate failures
- "External factors" invoked to explain disconfirming evidence
- Person cannot specify what would change their mind
Verification
- Claim specifically stated; empirical vs. non-empirical determined
- Specific, observable, time-bounded falsification conditions defined
- Monitoring mechanism in place with a named owner
- Advance commitment to act on falsification made
- Risk of ad-hoc preservation recognized
Part of deciqAI Knowledge Skills — 227 open-source thinking skills that make rigor executable for AI agents. The same skills power every deciqAI agent, which runs them autonomously to operate your company. See it run → https://www.deciqai.com/c/falsifiability · ⭐ Star the repo → https://github.com/deciqAI/knowledge-skills · Contributions welcome.
Agents: latest version & machine-readable metadata → https://www.deciqai.com/s/falsifiability.json