The Thundering Herd Problem
When everyone retries at once, and how jitter and backoff fix it
What you'll learn
- What a thundering herd is
- How it shows up in retries, caches and locks
- Spreading retries out with jitter and backoff
- Letting one request rebuild, and waking one waiter
In this video, we'll look at why a service can come back after an outage and fall over again right away. We'll use 10,000 clients that all retry after exactly five seconds to show how waiting clients rush in at the same moment and crush a recovering server.
We'll also look at the same problem in caches and in threads waiting on a lock, and the fixes: jitter, exponential backoff, letting only one request rebuild a cached value, and waking one waiter at a time.
By the end, you'll understand what causes a thundering herd and how to spread the load out.
