Designing Caching Architecture
1. Designing Cache Hierarchies
| Tier | Latency | Examples |
|---|---|---|
| Browser | 0ms | Cache-Control, ServiceWorker |
| CDN edge | 10–50ms | Cloudflare, CloudFront |
| Reverse proxy | 1–5ms | Varnish, NGINX |
| App in-process | µs | Caffeine, Guava, lru-cache |
| Distributed | ~1ms | Redis, Memcached |
| DB cache | ~ms | Buffer pool, materialized views |
2. Designing Cache-Aside Pattern
Cache-Aside (Lazy Loading)
- App reads cache; on miss, reads DB
- App writes value to cache with TTL
- On write, app updates DB then invalidates/updates cache
| Pro | Con |
|---|---|
| Resilient (cache failure ≠ outage) | Stale data possible; thundering herd |
3. Designing Write-Through Cache
| Aspect | Detail |
|---|---|
| Flow | App writes cache → cache writes DB synchronously |
| Consistency | Strong cache+DB |
| Latency | Higher write latency |
| Use | Read-heavy with consistent cache |
4. Designing Write-Behind Cache
| Aspect | Detail |
|---|---|
| Flow | App writes cache; cache flushes to DB async |
| Pro | Low write latency, batching |
| Con | Risk of data loss on cache crash; harder consistency |
| Use | Counters, analytics, bursty writes |
5. Designing Read-Through Cache
| Aspect | Detail |
|---|---|
| Flow | App always asks cache; cache loads from DB on miss |
| Pro | App code simpler |
| Con | Requires cache library / provider integration |
| Examples | Hibernate 2nd-level, AWS DAX |
6. Understanding Cache Eviction Policies
| Policy | Behavior | Best For |
|---|---|---|
| LRU | Evict least recently used | General workloads |
| LFU | Evict least frequently used | Skewed access |
| FIFO | Evict oldest insert | Streams, queues |
| TTL-based | Time-bound entries | Time-sensitive data |
| ARC / W-TinyLFU | Adaptive (Caffeine) | High hit ratio |
| Random | Cheap; OK at high hit rate | Memory-constrained |
7. Designing Cache Warming Strategies
| Strategy | When |
|---|---|
| Pre-load on deploy | Predictable hot keys |
| Replay logs | Warm new node from access log |
| Background refresher | Refresh entries before expiry |
| Shadow traffic | Mirror prod requests to new node |
8. Designing Cache Invalidation Strategies
| Strategy | Description |
|---|---|
| TTL | Time-based; simple but stale window |
| Explicit invalidate on write | Strong; risk of race |
| Write-through | Update cache atomically |
| Tag-based | Invalidate group via tag |
| Versioned keys | Bump version → effectively new key |
| CDC-driven | DB change events update cache |
Warning: "There are only two hard things in computer science: cache invalidation and naming things." Pick the simplest correct strategy for your consistency need.
9. Implementing Distributed Caching
| System | Strength |
|---|---|
| Redis Cluster | Data structures, pub/sub, persistence |
| Memcached | Simple K/V, multi-threaded, very fast |
| Hazelcast / Ignite | In-memory data grid, compute |
| DragonflyDB / KeyDB | Redis-compatible, multi-threaded |
| Sharding | Consistent hash across nodes |
10. Designing Cache Stampede Solutions
| Technique | Description |
|---|---|
| Request coalescing | Singleflight: 1 fetch per key, others wait |
| Lock + recompute | Distributed lock during recompute |
| Probabilistic early expire | Refresh before TTL with probability |
| Stale-while-revalidate | Serve stale; refresh in background |
| Jittered TTLs | Avoid synchronized expiration |
11. Designing Cache Consistency Strategies
| Strategy | Order of Ops | Risk |
|---|---|---|
| Update DB → invalidate cache | Write DB, DEL cache | Stale if reader sees old before DEL |
| Update DB → update cache | Atomic with txn outbox | Race in concurrent writes |
| Cache-aside double-delete | DEL, write DB, DEL again after delay | Reduces race window |
| CDC stream | Debezium → cache updater | Most consistent |
12. Designing Multi-Level Cache Architecture
Client ─▶ L1 (in-process Caffeine) ─▶ L2 (Redis) ─▶ DB
hit µs hit ~1ms
Invalidation: DB write → CDC → Redis update → pub/sub → L1 evict
| Level | Size | Hit Latency |
|---|---|---|
| L1 (local) | MB-GB | µs |
| L2 (distributed) | GB-TB | ~1ms |
| L3 (CDN/edge) | TB+ | 10–50ms |