jonnify·redis /redis-patterns/time-delay-priority/backoff-retry/jitter-thundering-herd

R4.04.2 · Backoff & retry — dive 2

Breaking the thundering herd.

A fixed backoff schedules every job that failed together for the same future instant, so they re-fire in one spike. Jitter spreads them: a random delay bounded by the backoff delay scatters the cohort across a window.

Grounding: EchoMQ.Backoff.delay_ms/2 · the {:jitter, inner} policy (echo/apps/echo_mq/lib/echo_mq/backoff.ex) — a uniform random delay in 0..inner_delay, the full-jitter form.

A cohort re-fires — with and without jitter. Hover or select a mode to see the spike collapse into a spread.
t = now + delay window
readout · Select a mode. Without jitter eight jobs stack at one instant; with full jitter they scatter across the window.

Static verdicts — no jitter: 8 jobs at one fire-time (a spike of 8) · full jitter: 8 jobs across the window (peak of 1 at any instant).

§1

The herd

delayed-queue.md.txt · Retry with Backoff

A dependency blinks. A hundred jobs fail in the same moment, each at attempt 3. A fixed exponential backoff prices the same delay for all of them — 4 × base — so all hundred are scheduled for the same future instant. When that instant arrives the promotion sweep moves all hundred back to pending at once, and they hit the dependency together: a second spike on a service that was already struggling.

The retry meant to relieve the dependency instead re-creates the original pressure. The problem is not the backoff value — it is that the value is identical for the whole cohort, so the cohort stays synchronized.

cohort
A set of jobs that failed at the same moment, sharing an attempt count.
thundering herd
The cohort re-firing together because a fixed backoff gave them one fire-time.
full jitter
A delay drawn uniformly from 0..delay, so the bound is the whole backoff value.
spread
The window the jittered fire-times occupy, replacing the spike.

A fixed backoff is correct per-job and wrong per-cohort: it relieves one job and synchronizes a thousand.

§2

Spreading the cohort

jitter · uniform in 0..delay

Jitter breaks the synchrony. Instead of scheduling each job at exactly delay, schedule it at a random point bounded by delay. A hundred jobs that failed together now carry a hundred different fire-times scattered across the window, and the promotion sweep releases them gradually. The dependency sees a spread, not a spike.

The cost is variance: a given job's exact wait is no longer predictable, only its bound. That is the trade — predictability for the absence of a herd — and for retry work it is almost always worth taking. Set the inner backoff below and read the spread width a full-jitter draw produces.

The spread width. Set the inner backoff delay; the readout shows the 0..delay window full jitter draws across, over a fixed eight-job sample.
4000

Static verdicts — inner delay 4000 ms → full-jitter bound 0..4000 ms; the fixed eight-job sample spans the window, peak of 1 at any instant (vs a spike of 8 unjittered).

Full jitter trades a precise fire-time for the absence of a herd: the bound is the backoff value, and the cohort spreads across 0..delay.

§3

Jitter as a policy wrapper

echo_mq/backoff.ex · {:jitter, inner}

In the real bus jitter is not a flag on the formula; it is a policy that wraps any inner policy. {:jitter, inner} computes the inner policy's delay, then draws a uniform random value bounded by it:

# EchoMQ.Backoff.delay_ms/2 — the :jitter clause (verbatim)
def delay_ms({:jitter, inner}, attempts) do
  bound = delay_ms(inner, attempts)                       # the inner curve's delay
  if bound <= 0, do: 0, else: :rand.uniform(bound + 1) - 1  # uniform in 0..bound
end

Wrapping the exponential policy — {:jitter, {:exponential, 100, 10_000}} — gives a curve that climbs and spreads: attempt 3's bound is 400, so the jittered delay is a uniform draw in 0..400. The inner delay is its bound, and the randomness is the point. This is the only non-deterministic policy in the vocabulary; :fixed and :exponential are pure functions of (policy, attempts). The interactive above demonstrates the spread over a fixed sample; the live draw uses :rand.uniform, so a real cohort's exact fire-times differ each run while their bound does not.

Notes on ValkeyEach jittered job is one ZADD at its own fire-time, so a single ZRANGEBYSCORE sweep releases the scattered cohort across the window rather than in one pull — valkey.io/topics/data-types.

the pattern

A fixed backoff synchronizes a failed cohort into one re-fire spike; full jitter draws each retry's wait uniformly in 0..delay, so the cohort re-fires across a window.

its EchoMQ application

EchoMQ.Backoff.delay_ms({:jitter, inner}, attempts) returns a uniform random delay bounded by the inner curve; EchoMQ.Jobs.retry/7 schedules each job at now + delay_ms, scattering the cohort across the schedule set.

Jitter wraps any curve. The inner policy sets the bound; the wrapper draws the spread.

§4

Recap — spread, do not stack

next → R4.04.3

A fixed backoff schedules a failed cohort for one instant, so it re-fires as a herd. Full jitter draws each retry's wait uniformly in 0..delay, scattering the cohort across a window. In the real bus jitter is a policy wrapper, {:jitter, inner} — the only non-deterministic policy in the vocabulary. The next dive shows the retry adds no new structure: it rides the same schedule set R4.01 built.

§5

References

sources · related

Sources

Related in this course