jonnifyechomq

Single-flight & jittered TTL · dive 02

Jittered
expiry

A cohort filled together never expires together.

A fixed TTL is a trap: a set of keys filled at the same millisecond expires at the same millisecond, and every caller that touches any of them gets a miss simultaneously — the thundering herd re-forms at the boundary, on a timer. expires_at/1 breaks the synchrony: each row's expiry is drawn from a uniform band around the declared TTL (ttl ± ttl·jitter), so a cohort's deaths are spread across time, not stacked at one tick. The sweeper reclaims dead rows on a fixed sweep_ms tick using :ets.select_delete with a guard on the expires_at field — no per-row timer, no extra process per entry, memory bounded by the declaration.

expires_at/1 The sweeper Jitter band Pattern & implementation References

expires_at/1

Every row inserted into L1 — by a flight completing or a put/3-4 call writing through — gets its expiry from expires_at/1. The function takes the table's declared spec, reads the monotonic clock in milliseconds, and adds the TTL plus a random offset drawn from the jitter band.

The declared jitter is validated at init/1 to be in 0.0..0.5. At 0.0 the spread is zero and every row gets exactly base. At 0.5 the spread is half the TTL, so a row may expire anywhere from 0.5·ttl to 1.5·ttl after insertion.

# EchoStore.Table — expires_at/1
# Returns the monotonic-ms timestamp at which this row should be
# considered dead. The jitter band is ± spread around base.
defp expires_at(spec) do
  base   = System.monotonic_time(:millisecond) + spec.ttl_ms
  spread = trunc(spec.ttl_ms * spec.jitter)   # how far from base to allow

  if spread == 0 do
    base     # jitter 0.0 → exactly TTL, no randomness (useful in tests)
  else
    # :rand.uniform(n) returns 1..n; shift to -spread..(spread-1)
    # so the expiry is base - spread .. base + spread (approximately)
    base + :rand.uniform(2 * spread + 1) - spread - 1
  end
end

The monotonic clock (:millisecond epoch local to this node) is the right choice here: it never leaps backwards on an NTP adjustment, so an expiry comparison (now < expires_at) is always valid within a single node's L1 table. L2 TTL is a separate Valkey PX argument and uses the server clock.

The sweeper

Jitter alone does not remove dead rows. ETS has no built-in TTL eviction: a row past its expires_at stays in the table until something deletes it. The sweeper closes the loop.

handle_info(:sweep) is armed at init/1 with Process.send_after(self(), :sweep, sweep_ms) and re-arms itself at the end of every pass. It runs :ets.select_delete with a match specification that guards on the expires_at field (index $1 in the three-element tuple {id, value, expires_at, version}) and deletes every row for which expires_at < now.

# EchoStore.Table — handle_info(:sweep)
# Deletes every L1 row whose expires_at is before now (monotonic ms).
# The match spec tuple is {id, value, expires_at, version};
# $1 binds expires_at; the guard is expires_at < now.
# Counts :swept (rows removed) and :sweeps (passes), then re-arms.
def handle_info(:sweep, state) do
  now = System.monotonic_time(:millisecond)

  removed =
    :ets.select_delete(state.name, [
      {{:_, :_, :"$1", :_}, [{:<, :"$1", now}], [true]}
    ])

  :counters.add(state.spec.counters, counter(:swept), removed)
  :counters.add(state.spec.counters, counter(:sweeps), 1)
  Process.send_after(self(), :sweep, state.spec.sweep_ms)   # re-arm
  {:noreply, state}
end

The sweeper bounds memory by the declaration: a table with sweep_ms: 1_000 and ttl_ms: 60_000 will never accumulate more than roughly max_size live rows older than the jittered TTL window. Dead rows are evicted on a regular clock, not by luck.

The jitter band: before and after

Set the TTL and jitter and compare two cohorts of rows: one with jitter, one without. Without jitter, every row in the cohort expires at the same tick — a perfect thundering herd at the boundary. With jitter the expirations spread across the band. The model runs the exact expires_at/1 arithmetic over a fixed cohort of 20 rows filled at the same millisecond t=0.

Jitter band · cohort expiry spread

10000
0.20
NO JITTER all 20 expire at t=10000ms WITH JITTER spread across band

Jitter trades a predictable single-tick expiry storm for a smooth drizzle across a band — a cohort filled together still expires close together, never all at once. The sweeper reclaims them across multiple ticks rather than in one pass.

Pattern & implementation

the pattern (jittered TTL)

Add a random offset to each item's TTL so a cohort filled together expires at different times, preventing the mass-miss (thundering herd) that forms at a shared boundary. Redis Patterns Applied · R1 teaches the TTL-staleness and early-refresh patterns; jitter is the simplest form of the same defense.

the implementation (EchoStore.Table)

expires_at/1 draws from base ± spread using :rand.uniform. The sweeper runs :ets.select_delete with a guard on the stored expires_at field on a fixed sweep_ms tick. No per-row timer, no extra process.

The pattern says: spread expirations. The implementation says: draw from a uniform band at insert time, then reclaim in one select-delete pass per tick.

References

Sources

Related in this course