Caching | System Design Interview Handbook

🪜

Where Caches Live

a read's journey through every layer

Caching isn't one thing in one place. A read passes through a stack of caches, each closer to the user and cheaper to hit than the last. The art is stopping the read as early as possible.

  request ─▶ ┌─────────────┐  hit → done (0 network)
             │  Browser     │  local, per-user
             └──────┬──────┘  miss ▼
             ┌─────────────┐  hit → served from the edge
             │  CDN         │  static/near-static (Networking Basics)
             └──────┬──────┘  miss ▼
             ┌─────────────┐  hit → skip app logic
             │  Gateway/edge│
             └──────┬──────┘  miss ▼
             ┌─────────────┐  hit → skip the DB   ← the big one:
             │  App cache   │  Redis / Memcached      Redis in front of the DB
             └──────┬──────┘  miss ▼
             ┌─────────────┐  DB internal caches (buffer pool)
             │  Database    │  the source of truth
             └─────────────┘

ℹ️

Each layer catches what the ones above missed. The browser and CDN handle static content (Networking Basics); the application cache — Redis or Memcached in front of the database — is the one you design most deliberately, and the rest of this page is mostly about it.

↩️

Cache-Aside

the default read pattern

The default and most common pattern: the application checks the cache, and on a miss reads the database and populates the cache itself. The cache sits beside the data path, and the app owns the logic.

  read(key):
    v = cache.get(key)
    if v is not None:  return v          ── HIT
    v = db.read(key)                     ── MISS
    cache.set(key, v, ttl)               populate for next time
    return v

Simple and resilient: if the cache is down, reads fall through to the database — degraded, not broken. Two things to name. Only requested data is cached (lazy population), so the first read of anything is always a miss. And if the cache dies, every read hits the database at once — which is why the failure modes below matter.

✍️

Write Strategies

keeping cache and store in step

Reads are the easy half. When data changes, you choose how the write touches cache and database, trading write latency against consistency and durability.

Strategy	How	Trade
Write-through	Write cache and DB together, synchronously	Cache always fresh; every write pays both hops
Write-behind	Write cache now, flush to DB async later	Fast writes; data loss if the cache dies before flush
Write-around	Write DB only; cache fills on later reads	Avoids caching write-only data; first read misses

💡

Match the strategy to the access pattern. Write-through when reads follow writes closely and freshness matters. Write-behind when write throughput is the constraint and you can tolerate a small loss window. Write-around for data written far more than it's read, so you don't pollute the cache with entries nobody reads back (Tradeoffs).

🗑

Eviction & Invalidation

the two hard things, earned

Two separate questions. Eviction: the cache is full, what do you drop? Invalidation: the underlying data changed, how does the cache stop serving stale copies? Eviction is a capacity policy; invalidation is a correctness problem, and it's genuinely hard.

Eviction policies what to drop

LRU — drop least recently used. The default; assumes recent = likely again.
LFU — drop least frequently used. Better for stable hot sets.
TTL — expire after a fixed time, regardless of use.

Size the cache with the 80/20 rule — roughly 20% of data serves 80% of reads (Estimation).

Invalidation approaches how to stay fresh

TTL expiry — accept staleness up to the TTL. Simplest, no write coordination.
Explicit — delete the key on write. Fresh, but every writer must remember.
Event-driven — invalidate via a CDC stream off the DB log, so nothing is forgotten (CDC).

⚠️

The reason "there are only two hard things, and one is cache invalidation" is earned: the moment you cache, you have two copies that can disagree, and every write must decide how to reconcile them. TTL trades freshness for simplicity; explicit invalidation trades simplicity for freshness and breaks the instant one write path forgets to invalidate.

💥

Failure Modes

the four that come up by name

Caches fail in specific, named ways, and each has a standard mitigation. Knowing them by name is a strong signal in an interview.

  STAMPEDE / thundering herd
    a hot key expires → thousands of requests miss at once → all hit the DB
    fix: lock so one request rebuilds while others wait; request coalescing;
         probabilistic early refresh before expiry

  HOT KEY
    one key gets so much traffic it overloads a single cache node
    fix: replicate the key across nodes; add a local in-process cache

  PENETRATION
    requests for a key that DOESN'T exist always miss and hit the DB
    fix: cache the negative result; a Bloom filter to reject known-absent keys

  AVALANCHE
    many keys expire together (or the cache tier restarts) → mass miss storm
    fix: jitter the TTLs so they don't expire in sync; warm the cache on start

Failure	Cause	Mitigation
Stampede	Hot key expires, all miss at once	Locking, request coalescing, early refresh
Hot key	One key overloads one node	Key replication, local caching
Penetration	Missing keys always miss	Negative caching, Bloom filter
Avalanche	Mass simultaneous expiry / cold restart	Jittered TTLs, cache warming

ℹ️

Notice the shared root: a cache miss is only cheap when it's rare. Every failure here is a moment when misses become simultaneous, and the fixes all work by spreading them out in time (jitter, early refresh) or collapsing them into one (locking, coalescing). The database behind the cache was never sized for the uncached load (Resilience Patterns).

📋

Quick Reference

strategies and failures on one card

Write strategy	Fresh?	Best for
Write-through	Always	Read-after-write, freshness matters
Write-behind	Eventually	Write-heavy, tolerates a loss window
Write-around	On next read	Write-mostly data
Cache-aside (read)	Up to TTL	The default read path

Failure	Mitigation
Stampede	Locking, coalescing, probabilistic early refresh
Hot key	Key replication, local cache
Penetration	Negative caching, Bloom filter
Avalanche	Jittered TTLs, cache warming

ℹ️

The one-liners to keep ready. A read walks a stack of caches — stop it as early as you can. Cache-aside is the default; the app populates on miss and degrades gracefully if the cache dies. Pick a write strategy by access pattern. Invalidation is the hard part because caching means two copies that can disagree. And name the failure modes — stampede, hot key, penetration, avalanche — because a cache miss is only cheap when it's rare.