The Cache · module 02
One fill
per herd
A cache that does not stampede and does not grow without bound.
Module 01 built the declared near-cache and showed how a read resolves across the two
layers. Module 02 is the second half: the discipline that keeps it cheap under a herd
and bounded under pressure. Concurrent misses on the same key coalesce onto a single
in-flight load — one fill, not a thousand. Expiry is jittered so a cohort never
dies in step, and a sweeper reclaims dead rows on a fixed tick. And when the table is full and no
expired rows exist, a fill still serves its caller from L2 and the loader: the cache
degrades to pass-through, it never fails. All three laws are in
EchoStore.Table, real shipped code.
A herd, one flight
Pick how many callers miss simultaneously on the same key and watch the owner's mailbox
coalesce them into a single loader call. The :coalesced counter
records each waiter that did not start its own flight. Pick a path to see the call sequence
and the counter update.
Herd coalescing · pick a caller count
The owner's mailbox is the serialization point — the first {:fill,
id} starts a flight; every other appends. The loader runs once; every waiter reads that one
answer. The :coalesced counter records every saved loader call.
The dives
Three dives, each a law from EchoStore.Table: the coalescing
flight, the jittered clock, and the graceful degrade under pressure.
One fill per herd
handle_call({:fill, id}) coalesces concurrent misses onto one flight; the spawn_monitor'd task GETs, loads, and replies to every waiter.
Jittered expiry
expires_at/1 draws each row's death from ttl ± spread; the sweeper reclaims dead rows on a fixed sweep_ms tick.
The full cache degrades
insert/4 tries to reclaim expired rows before skipping; a skip is a pass-through, not a failure. stats/1 makes the rate observable.
References
Sources
- Erlang/OTP — the ets module — the read-concurrent L1 table the flight fills.
- Valkey — Cluster specification — the
{table}hashtag and one-slot discipline. - Valkey — GET — the L2 probe a flight issues on a miss.
- Valkey — SET — the framed
version <> valuewrite after a fill. - Helland — Life Beyond Distributed Transactions — the entity addressed by a key, cached close to its use.
- King — Announcing Snowflake — the mint-time id every cached value is framed with.
Related in this course
- The Cache — the chapter landing; module 01 is the entry point.
- Cache-aside, two layers — module 01: the declared tiers and the read path.
- The Protocol — the keyspace the
ecc:prefix stands beside. - The Queue — the other pillar over the same wire.
- The Branded Component System · the Store — the manuscript chapter this pillar realizes.
- Redis Patterns Applied · R1 Caching — the patterns this module applies in depth.