Resilience Patterns (Circuit Breaker/Retry/Bulkhead)

Resilience patterns are design techniques that keep a service responsive when the dependencies it calls slow down or fail. The three most common are the circuit breaker, retry with exponential backoff, and the bulkhead, and together they stop a local fault from cascading into a system-wide outage.

A circuit breaker wraps a remote call and watches its failure rate. It has three states: closed (calls flow normally), open (calls fail fast without touching the dependency), and half-open (a few trial calls test whether the dependency has recovered). When failures cross a threshold the breaker trips open, giving the struggling dependency room to recover and returning an immediate fallback to the caller instead of a slow timeout.

Retry with exponential backoff re-attempts a failed call but waits progressively longer between attempts (for example 1s, 2s, 4s) so a struggling dependency is not hammered. Adding jitter (a randomized delay) prevents many clients from retrying in lockstep and creating a synchronized spike. Retries are only safe when the operation is idempotent; otherwise a retry can duplicate a side effect such as a payment.

A bulkhead isolates resources so one failure cannot sink the whole ship. By giving each dependency its own connection pool or thread pool, a slow dependency can exhaust only its own partition while calls to healthy dependencies keep flowing.

These patterns pair naturally with observability: you cannot tune a breaker threshold or a retry budget without metrics on failure rates and latency.

Retries trade extra load for a higher success rate, and without backoff, jitter, and a capped attempt count they cause retry storms that amplify an outage. A circuit breaker adds tuning burden: thresholds set too tight trip on noise, and set too loose they never protect anything. Bulkheads improve isolation but reserve capacity per partition, so total utilization drops and each pool must be sized. All three add complexity and should be applied where a dependency is genuinely at risk, not everywhere.

• When a dependency sits on your critical path, say you would wrap it in a circuit breaker with a sensible fallback.
• Whenever you mention retries, immediately add "with exponential backoff and jitter" and note that the target must be idempotent - interviewers listen for exactly that pairing.
• Reach for a bulkhead when one shared thread pool serves several dependencies of differing reliability.