jonnifyechomq

Single-flight & jittered TTL · dive 03

The full cache
degrades

A full table becomes a pass-through. It never refuses a read.

When insert/4 finds the ETS table at max_size, it tries to make room first: reclaim/1 runs the same :ets.select_delete the sweeper uses, but right now. If that frees at least one slot, the insert proceeds. If the table is genuinely full of live rows, insert/4 counts :full_skips and returns :skip. The caller still has its value — from the loader or from L2 — and returns it. A full cache is a pass-through, not a failure. The stats/1 surface exposes the live counters so the degrade rate is observable without a trace. And every row already carries the mint-time version that module 03 (Coherence) rides — a one-line door, not a lesson.

insert/4 and reclaim/1 stats/1 Reclaim vs skip Pattern & implementation References

insert/4 and reclaim/1

Every time a flight completes or put/3-4 writes through, the result passes to insert/4. The function checks the current ETS table size and makes one of three choices in order:

  1. Fast path: size < max_size — insert directly.
  2. Reclaim path: the table is full, but reclaim/1 finds and deletes expired rows. If it freed at least one slot, insert.
  3. Skip path: the table is full of live rows. Count :full_skips, return :skip.

A skip is not an error. The caller already received the value (from the loader or from L2). The next read will miss L1 (because the insert was skipped), fall through to L2, and get the value there. Under sustained pressure the table operates as a straight-through L2 proxy.

# EchoStore.Table — insert/4
# Three-branch decision: insert, reclaim-then-insert, or skip.
# The ETS row shape is {id, value, expires_at, version}.
defp insert(state, id, value, version) do
  size = :ets.info(state.name, :size)

  cond do
    # Fast path: room available — insert directly under the jittered expiry.
    size < state.spec.max_size ->
      :ets.insert(state.name, {id, value, expires_at(state.spec), version})

    # Reclaim path: full, but expired rows may have accumulated since the
    # last sweep. Run select_delete right now; if it freed room, insert.
    reclaim(state) > 0 ->
      :ets.insert(state.name, {id, value, expires_at(state.spec), version})

    # Skip path: genuinely full of live rows. Count the skip; the caller
    # still has the value — this is a pass-through, not a failure.
    true ->
      :counters.add(state.spec.counters, counter(:full_skips), 1)
      :skip
  end
end

# reclaim/1 — on-demand sweep: the same select_delete the sweeper runs,
# called inline when the table is at max_size. Returns row count deleted.
defp reclaim(state) do
  now = System.monotonic_time(:millisecond)
  :ets.select_delete(state.name, [{{:_, :_, :"$1", :_}, [{:<, :"$1", now}], [true]}])
end

reclaim/1 is the same match specification as the scheduled sweeper. It runs in the caller's process via the GenServer call, so it does not race with the sweeper — the owner serializes both. Together they ensure dead rows are removed both on the fixed tick and on-demand when the table is under pressure.

stats/1

stats/1 returns the live counter snapshot and the current ETS table size. The counters are updated atomically throughout the GenServer by :counters.add; stats/1 reads them all in one pass. The returned map covers the eight counters taught in this module plus the live :size:

# EchoStore.Table — stats/1
# Returns a map of all counter values plus the live ETS table size.
# Counters are defined as [{name, index}, …] in @counters at module top.
def stats(name) do
  {:ok, spec} = EchoStore.spec(name)

  @counters
  |> Map.new(fn {k, i} -> {k, :counters.get(spec.counters, i)} end)
  |> Map.put(:size, :ets.info(name, :size))
end

# The @counters definition at module top (indices for :counters.add/get):
# hits:1  misses:2  fills:3  l2_hits:4  coalesced:5
# swept:6  full_skips:7  sweeps:8  coh_applied:9  coh_stale:10
# stats/1 returns indices 1–10 plus :size from :ets.info.
# coh_applied and coh_stale belong to the coherence law (module 03).
hits
L1 answers
misses
fell through
fills
loader ran
l2_hits
found in L2
coalesced
saves on fill
swept
rows reclaimed
full_skips
pass-throughs
sweeps
tick passes

The full_skips / (fills + full_skips) ratio is the fraction of fills that did not land in L1 — the cache's degrade rate. A sustained non-zero rate signals that max_size is too small or the TTL too long relative to the working set.

Reclaim vs skip: the three-branch decision

Pick a scenario to step through insert/4. The model runs the exact three-branch decision over a fixed table of max_size = 10 with a range of current sizes and expired-row counts. Read the branch taken and the counter update.

insert/4 · three branches, max_size = 10

insert/4 size = ets.info(:size) size < max ets.insert :ok reclaim > 0 reclaim → insert :ok full & no expired :full_skips++ :skip (pass-through)

The skip path is not an error path. The caller has the value; the table is not storing it in L1 right now. The :full_skips counter is the signal — not an exception, not a circuit-break.

Pattern & implementation

the pattern (graceful degradation)

A bounded cache that cannot accept a new entry should degrade gracefully — serve the read from the backing store, not refuse it — so the service stays available under memory pressure. The cache is an optimization; it must never be the critical path.

the implementation (EchoStore.Table)

insert/4 tries three paths in order: insert, reclaim-then-insert, skip and count. A skip means the caller proceeds with the value it already has from the loader or from L2. stats/1 makes the degrade rate visible as :full_skips.

Degrade, not fail. The cache's contract is to accelerate reads, not to gate them — so a full table hands off gracefully to the layer beneath it.

Coherence — the version on every row

Each row in the ETS table is {id, value, expires_at, version}. The 14-byte version is the mint-time branded id stored alongside the value. Module 03 (Coherence — newer wins on the Bus) rides that field: when a coherence message arrives with a newer version for an id, the row is evicted with a single comparison, no coordinator. The version is already there; the third module teaches what to do with it.

References

Sources

Related in this course