System Design Fundamentals · Scaling Fundamentals
Caching Strategies
Caching is the single highest-leverage technique in this entire guide: it's usually the difference between a database that falls over at 10K reads/sec and one that comfortably serves 500K. The idea is simple — keep a copy of frequently-read data somewhere much faster than the source of truth.
Where to cache
Client → CDN → Load Balancer → App-level cache → Database
(Ch. 4) (Redis/Memcached) (source of truth)
Caching can happen at every layer of this chain — a CDN cache for static assets (Chapter 4), an in-memory cache inside the application process, or a shared cache like Redis in front of the database. This section focuses on that last one, since it's the one you reach for most often.
Cache invalidation strategies
| Strategy | How it works | Trade-off |
|---|---|---|
| Cache-aside | App reads cache; on miss, reads DB and populates cache | Simple, but first request after expiry is always slow |
| Write-through | App writes to cache and DB together | Cache always fresh, but write latency includes both |
| Write-behind | App writes to cache immediately, DB is updated asynchronously | Fastest writes, but risk of data loss if the cache dies before flushing |
Cache-aside is the default for a reason — it's the simplest to reason about and fails safely (a cache outage degrades to "slow," not "wrong" or "data loss").
Eviction: what happens when the cache is full
The dominant policy is LRU (Least Recently Used) — evict whatever hasn't been touched in the longest time. It's a good default because real-world access patterns are usually skewed (a small set of "hot" keys account for most traffic — this is the same 80/20 shape you'll see in almost every caching discussion).
A cache is a trade of consistency for speed. The moment you add one, you've accepted that a reader can briefly see stale data. If your system genuinely cannot tolerate that (a bank balance mid-transfer, for example), that's a real signal the cached value shouldn't be the one used for that specific decision — even though the rest of the system caches freely.