Load Balancing — System Design Fundamentals

The moment you have more than one server, something has to decide which server handles which request. That's a load balancer's entire job — but how it decides has real consequences.

                    ┌──────────────┐
        requests →  │ Load Balancer │
                    └──────┬───────┘
                 ┌─────────┼─────────┐
                 ↓         ↓         ↓
            ┌────────┐┌────────┐┌────────┐
            │ Server1││ Server2││ Server3│
            └────────┘└────────┘└────────┘

Common algorithms

Algorithm	How it decides	Best for
Round robin	Cycles through servers in order	Uniform request cost, stateless servers
Least connections	Sends to whichever server has the fewest active connections	Requests with variable duration
Weighted round robin	Round robin, but bigger servers get proportionally more	Heterogeneous hardware
Consistent hashing	Hashes a request key (e.g. user ID) to a server	Session affinity, cache locality

Layer 4 vs. Layer 7

Layer 4 (transport) balances based on IP/port only — fast, but blind to the actual request content.
Layer 7 (application) reads the HTTP request itself — can route /api/video and /api/text to different backend pools, terminate TLS, and do smarter health checks. Slower per-request, but far more capable. Most production setups (e.g. behind an Nginx or a cloud load balancer) run Layer 7.

The failure mode people forget

A load balancer that doesn't do health checks will happily keep sending traffic to a dead server. Every real load balancer needs to periodically probe each backend (a lightweight /health endpoint is standard) and pull unhealthy nodes out of rotation automatically — otherwise the load balancer itself becomes the outage.