Design the failure path before production designs it for you.
Reliable systems are not defined by the happy path. They are defined by what they preserve, expose and recover when responses disappear, state becomes stale, two actors race or dependencies only partially succeed.
Failure modes we design for
Timeout after effect
The external change may be complete even when the caller receives no response. Retry must be based on effect evidence, not timeout alone.
Duplicate retry
One-use authority and postcondition evidence reduce the chance that an ambiguous first attempt becomes an accidental second effect.
Stale authority
Policy, entitlement, capability or resource state can change between decision and execution. Stale assumptions should fail visibly.
Concurrent actors
Two valid actors can race against the same resource. Concurrency is a correctness problem, not only a performance problem.
Partial effect
A multi-step operation can stop after changing only part of the world. Recovery needs explicit state and bounded reconciliation.
Dependency drift
Providers, schemas, routes and capabilities change. Discovery and compatibility evidence must not silently become permanent truth.
Operational rule
What resilience evidence should show
- what state is durable across restart;
- which actions are replay-safe and which are not;
- how stale authority is rejected;
- how unresolved operations are surfaced;
- how recovery avoids deleting user-owned state;
- which native/device/environment tests are still pending.