Make Operations Idempotent
Design operations so they converge to the correct state regardless of how many times they run or where they start from. Every state-mutating operation should answer: "What happens if this runs twice? What happens if the previous run crashed halfway?"
Why: Commands, lifecycle operations, and processing loops run where crashes, restarts, and retries are normal. If partial state changes the next run's outcome, every restart becomes a debugging session.
The pattern:
- Convergent startup: scan for existing state, clean stale artifacts, adopt live sessions
- Content-based cleanup: compare by content equivalence, not creation order
- Self-healing locks: use PID-based stale lock detection
- Idempotent scheduling: failed work respawns cleanly, fresh input regenerated after each cycle
For generated reports, indexes, and calibration files, rebuild from attributed source records and replace the generated output. Preserve source facts, append-only history, and human annotations. Report missing inputs rather than inventing samples or carrying stale derived values forward.
The test:
- What happens if this runs twice in a row?
- What happens if the previous run crashed at every possible point?
- Does re-execution converge to the same end state?
If any answer is "it depends on what state was left behind," the operation needs a reconciliation step.