Working with Distributed Caching
1. Understanding Cache Strategies
| Strategy | Read Path | Write Path |
|---|---|---|
| Cache-aside (lazy) | App reads cache; on miss, load from DB & populate | App writes DB; invalidate cache |
| Read-through | Cache loads from DB transparently | App writes DB |
| Write-through | Cache always queried first | Cache writes DB synchronously |
| Write-behind | Cache always queried | Cache writes DB async (risk of loss) |
| Refresh-ahead | Predictively reload before TTL | — |
2. Implementing Read-Through Caching
Example: Caffeine + LoadingCache
LoadingCache<String, User> cache = Caffeine.newBuilder()
.maximumSize(10_000)
.expireAfterWrite(Duration.ofMinutes(10))
.recordStats()
.build(key -> userRepo.findById(key));
User u = cache.get("user:42"); // loads from DB on miss3. Implementing Write-Behind Caching
| Property | Detail |
|---|---|
| Pros | Low write latency; write coalescing |
| Cons | Data loss on cache crash; ordering issues |
| Use cases | High-write metrics, counters |
| Mitigations | Persist queue (Redis Streams), shorter flush interval |
4. Implementing Cache Invalidation Strategies
| Method | Detail |
|---|---|
| TTL | Auto-expire after N seconds |
| Explicit DELETE | App invalidates on write |
| Versioned keys | user:42:v3 bumped on update |
| Event-driven | Subscribe to DB CDC; invalidate |
| Write-through | Update cache as part of write |
Note: "There are only two hard things in CS: cache invalidation and naming things." — Phil Karlton
5. Handling Cache Coherence
| Pattern | Detail |
|---|---|
| Single source of truth | One cache shard owns key |
| Pub/sub invalidation | Broadcast deletes (Redis CLUSTER, ElastiCache) |
| Versioning | Check version on read |
| Quorum reads | Multi-replica caches |
6. Implementing Distributed Cache Partitioning
| Method | System |
|---|---|
| Consistent hash | Memcached client (Ketama), Redis Cluster |
| Hash slots | Redis Cluster (16384 slots) |
| Client sharding | App hashes key → instance |
| Proxy | Twemproxy, Envoy Redis filter |
7. Understanding Cache Stampede Problem
| Cause | Mitigation |
|---|---|
| Many clients miss simultaneously | — |
| Solution: Lock / mutex | Single loader per key (singleflight) |
| Solution: Probabilistic early refresh | XFetch algorithm |
| Solution: Stale-while-revalidate | Serve stale; async refresh |
8. Implementing Cache Warming
| Trigger | Detail |
|---|---|
| Startup | Preload hot keys from DB |
| Scheduled | Refresh top-N every hour |
| Replay logs | Replay last hour's reads |
| Predictive | ML-based prefetch |
9. Implementing Cache Eviction Policies
| Policy | Behavior | Best For |
|---|---|---|
| LRU | Evict least recently used | General purpose |
| LFU | Evict least frequently used | Stable hot set |
| FIFO | Evict oldest insertion | Streaming workloads |
| Random | Random eviction | Cheap; near-LRU |
| TinyLFU / W-TinyLFU | Frequency sketch + LRU window | Caffeine default |
| ARC | Adaptive between recency & frequency | Mixed workloads |
10. Handling Cache Penetration and Breakdown
| Problem | Cause | Solution |
|---|---|---|
| Penetration | Queries for non-existent keys bypass cache | Cache negative results; bloom filter pre-check |
| Breakdown | Hot key expires; flood to DB | Mutex / singleflight; never expire hot keys |
| Avalanche | Many keys expire simultaneously | Jittered TTLs; staggered refresh |