Phase 6: Scalability & Reliability
Explore the concepts of Scalability & Reliability.
Rate Limiting (Token Bucket/Leaky Bucket/Sliding Window)
Rate limiting caps how many requests a client can make in a given time window, protecting a service from overload, abuse, and runaway cost. The token-bucket and leaky-bucket algorithms are the two classic mechanisms for enforcing those limits. 🧠 Mental model: Token bucket = a parking meter. It fills up with coins (tokens) at a steady rate; each request costs a coin. If you're out of coins, you wait. Leaky bucket = a funnel. No matter how fast you pour water in, it drips out at a constant rate.
Resilience Patterns (Circuit Breaker/Retry/Bulkhead)
Resilience patterns are design techniques that keep a service responsive when the dependencies it calls slow down or fail. The three most common are the circuit breaker, retry with exponential backoff, and the bulkhead, and together they stop a local fault from cascading into a system-wide outage. 🧠 Mental model: Circuit breaker = a fuse box that trips to protect your house. Retry with backoff = knocking on a door, then waiting longer between each knock so you don't annoy the neighbor. Bulkhead = watertight compartments on a ship - one flooded room doesn't sink the whole vessel.
Failover & Redundancy
Failover is the automatic switching to a redundant or standby computer server, system, hardware component or network upon the failure or abnormal termination of the previously active application. Redundancy is the duplication of critical components or functions of a system with the intention of increasing reliability of the system, usually in the form of a backup or fail-safe. 🧠 Mental model: Think of an airplane with multiple engines. If one engine fails, the other can still fly the plane. Redundancy is having two engines; failover is the pilot (or autopilot) relying on the second engine when the first one dies.
Deployment Strategies
Deployment strategies are the techniques for shipping a new version of a service without taking it down or risking a bad release on all users at once. The main patterns - rolling, blue-green, canary - differ in how they shift traffic from old to new, and feature flags decouple "deployed" from "released" so you can turn behavior on and off independently of the binary. 🧠 Mental model: Opening a remodeled restaurant. Rolling = renovate a few tables each night while staying open. Blue-green = build an identical second dining room and flip the "open" sign to it overnight. Canary = seat a handful of trusted regulars in the new room first and watch their reactions before inviting everyone.
Auto-scaling
Auto-scaling automatically adjusts the number of active servers or computing resources in a system based on current load. It ensures the system has enough capacity to handle traffic spikes (scale out) and saves money by shutting down idle resources during quiet periods (scale in). 🧠 Mental model: Think of a supermarket checkout area. If lines get too long, the manager opens more registers (scaling out). When the rush is over and cashiers are standing idle, the manager closes registers and sends staff on break (scaling in).
Health Checks & Heartbeats
Health checks and heartbeats are the mechanisms by which a distributed system monitors the status of its components. They allow load balancers to stop sending traffic to broken servers and allow orchestrators to replace failed nodes automatically. 🧠 Mental model: A heartbeat is like a night watchman clicking a radio button every 60 seconds to say "I'm still awake." A health check is like a supervisor calling the watchman and asking, "Are you awake, and are all the doors locked?"
Geospatial Indexing
Geospatial indexing organizes location data so that "find everything near this point" is fast, instead of scanning every row and computing distance. The core trick is to reduce two dimensions (latitude, longitude) to a one-dimensional key that preserves proximity, so nearby places end up near each other in the index. Geohash, quadtrees, and Google's S2 are the dominant schemes. 🧠 Mental model: Imagine folding a paper map into ever-smaller labeled squares. A geohash is the label: the more letters it shares with your square, the closer it is. To find neighbors, you just look at squares whose labels start the same way - no need to measure distance to every city on Earth.