How do you prevent a cache stampede?
Interviewer Expectations for this Question
01
Core Competency
Assesses fundamental understanding of Caching Strategies conventions, runtime behavior, and memory/performance considerations.
02
Evaluation Criteria
Hiring managers look for precision, avoidance of ambiguous jargon, and ability to explain trade-offs under real production conditions.
Comprehensive Model Answer
Verified Solution
A stampede (thundering herd) happens when a popular key expires or is cold, and many concurrent requests all miss and hit the database at once, potentially overwhelming it.
Preventions:
- Locking or single-flight: the first request loads the value while others wait for it or return a short-lived stale value. Libraries and
singleflighthelpers implement this. - Probabilistic early expiration: refresh the key slightly before it expires, with jitter, so requests do not converge on the same instant.
- Stale-while-revalidate: serve the stale value immediately and refresh in the background.
- TTL jitter: add randomness to expiry so many keys do not expire together (avoiding an avalanche).
- Request coalescing at the application or cache layer.
const lock = await mutex.acquire(key);
if (lock) { value = await loadAndSet(key); lock.release(); }
else { value = await cache.getStale(key); }
Also warm critical keys on deploy and after restarts, and consider serving a degraded but useful response rather than blocking every request.
Candidate Response Strategy & Interview Tips
- Start with a concise one-sentence summary: Deliver a direct, confident answer first before expanding into nuances.
- Demonstrate real-world trade-offs: Discuss where this approach excels and when you would avoid it in production systems.
- Discuss complexity & edge cases: Proactively explain time/space complexity or boundary conditions (null values, scale limits).
- Prepare for interviewer follow-ups: Technical hiring panels frequently probe deeper into concurrency, backward compatibility, or alternative libraries.