EchoMQ / The Cache / Workshop

← the depth behind redis-patterns R1 · Caching

The Cache · workshop · end to end

A codemojex near-cache

Four moves, two layers, one game read — declared, hit, herd-proofed, and kept honest across nodes.

This page closes the Cache pillar by building one thing out of all three modules: a codemojex game read near-cached. On every guess, the scorer needs the game and its secret — the hottest read in the game. So we declare a cache in the directory (Codemojex.Tables), hit it at ETS speed with the kind gate at the door (EchoStore.Table.fetch/3), survive a herd with one fill and a jittered clock, then invalidate from another node with a message about a name (EchoStore.Coherence). The thread through all four is one branded id — the entity's own 14-byte id is the value's version, the coherence clock, and the key the gate checks. Step through the build on the right.

1 · Declare 2 · Hit 3 · Survive 4 · Invalidate The cache, built References

The build · step through the four moves

Move 1 — declare a codemojex cache

The first law of the near-cache: the cache is declared, not discovered. A codemojex cache exists only because Codemojex.Tables registered its full specification at start. The directory is the roster of every declared cache on the node, and a cache absent from it does not exist. codemojex declares three: :cm_games (kind GAM, a game and its secret), :cm_emojisets (kind EMS, the emoji layout), and :cm_sessions (kind SES, the first mutable one).

# echo/apps/codemojex — Codemojex.Tables (the declaration site)
# Three declared L1-over-L2 caches in front of Postgres on the scoring hot path.
# Each registers a full spec; a cache absent from the directory does not exist.
children = [
  # the games near-cache: a game + its secret, read on every guess score.
  Supervisor.child_spec(
    {EchoStore.Table,
     name: :cm_games,        # the declared cache name (the L1 :ets table name)
     kind: "GAM",            # the namespace the kind gate enforces on every id
     loader: &load_game/1,   # the 1-arity fall-through to the system of record
     coherence: :none,       # immutable for the game's life — never goes stale
     ttl_ms: 600_000,
     max_size: 50_000},
    id: :cm_games_table
  )
  # … :cm_emojisets (EMS) and :cm_sessions (SES, :tracking) declared the same way.
]
# started :rest_for_one under the EchoStore.Directory, so a directory restart
# cascades to the tables and they re-register — the roster cannot silently empty.

Under the hood the table's init/1 does the registration the directory monitors — and the monitor is the point: a crashed cache leaves the roster the instant it leaves the node.

# echo/apps/echo_store — EchoStore.Table.init/1 (the declared registration)
:ets.new(name, [:set, :public, :named_table, read_concurrency: true])  # the L1 tier
spec = %{kind: kind, ttl_ms: ttl_ms, jitter: jitter, max_size: max_size,
         sweep_ms: sweep_ms, coherence: coherence, counters: counters}
:ok = EchoStore.Directory.register(name, spec, self())  # monitored — a :DOWN drops the row

An operator enumerates every declared cache with EchoStore.tables/0 — [{:cm_emojisets, spec}, {:cm_games, spec}, {:cm_sessions, spec}], sorted — and reads one spec with EchoStore.spec(:cm_games). The L2 key the cache will address is EchoStore.Keyspace.key("cm_games", game_id) → ecc:{cm_games}:GAM… — a fresh prefix beside emq:, the {cm_games} hashtag landing every key of this cache on one of 16384 Valkey Cluster slots, with the id shape-checked before the key is composed.

A codemojex cache is a written-down thing — a kind, a TTL, a coherence mode, registered and watched. There are no ambient caches, only declared ones.

Move 2 — hit at ETS speed

With the table declared, the scorer reads the game through Codemojex.Cache.fetch_game/1 — the read-hot seam over EchoStore.Table.fetch/3. The read path never enters the owning process. A hit is a caller-side :ets.lookup in the scorer's own process, so reads scale with schedulers, not with one GenServer's mailbox.

# echo/apps/codemojex — Codemojex.Cache.fetch_game/1 (the read-hot seam)
# Read a game through the L1/L2 cache, falling back to the system of record. The
# cached value is term_to_binary(game_map); EchoStore frames it with the game's
# own id as the version. A direct Postgres fallback covers the boot window only.
def fetch_game(game_id) do
  case EchoStore.Table.fetch(:cm_games, game_id) do
    {:ok, bin, _source} when is_binary(bin) -> :erlang.binary_to_term(bin)
    _ -> Codemojex.Store.game(game_id)
  end
end

fetch/3 runs the kind law first, then tries the layers in order — a hit answers in the caller's process and never touches the owner; a miss is the only owner call.

# echo/apps/echo_store — EchoStore.Table.fetch/3 (the cache-aside read)
# 1. find the cache in the directory (absent → {:error, :no_such_cache});
# 2. GATE the id: 14 bytes, namespace == the table's declared kind, BrandedId.valid?
#    — a wrong-namespace id is refused at the door, before either layer is touched;
# 3. a caller-side :ets.lookup — a live row (now < expires_at) is {:ok, value, :hit}
#    and NEVER touches the owner; a miss is the only owner call: {:fill, id}.
def fetch(name, id, timeout \\ 10_000) do
  case EchoStore.spec(name) do
    :error -> {:error, :no_such_cache}
    {:ok, spec} ->
      with :ok <- gate(spec.kind, id) do
        now = System.monotonic_time(:millisecond)
        case :ets.lookup(name, id) do
          [{^id, value, expires_at, _v}] when now < expires_at ->
            {:ok, value, :hit}                # in the caller's process — no owner, no wire
          _ ->
            GenServer.call(name, {:fill, id}, timeout)  # the only path that consults the owner
        end
      end
  end
end

The answer is tagged with its source: :hit (L1, in the caller's process), :l2 (found in the shared Valkey), or :fill (loaded through the declared loader and written to both layers). The gate/2 is the series' oldest law riding into the cache unchanged — a PLR… id handed to the :cm_games cache (kind GAM) is {:error, :kind}, refused before any key is composed. And the L2 frame is self-describing: a cached value is stored as version <> value and split as <<version::binary-14, value::binary>>, so the game's own GAM id rides with the bytes.

A hit is a lookup in your own process, gated by the brand. The owning GenServer is consulted only when L1 has nothing — the read scales with schedulers, not a mailbox.

Move 3 — survive a herd

A cache that serves a hot read must not stampede when a popular game's row expires, and must not grow without bound. The second law holds on the miss path: one fill per herd. When a thousand guesses arrive for one game the instant its row expires, the owner runs one flight, not a thousand.

# echo/apps/echo_store — EchoStore.Table.handle_call({:fill, id}) (single-flight)
# Re-check L1 first — the race may have been won between the caller's miss and this
# call. Then: if a flight for this id already exists, append the caller to its
# waiters and count :coalesced — start NO second flight. Else launch one flight.
case Map.fetch(state.flights, id) do
  {:ok, {ref, waiters}} ->
    :counters.add(state.spec.counters, counter(:coalesced), 1)
    {:noreply, put_in(state.flights[id], {ref, [from | waiters]})}   # join the herd
  :error ->
    ref = launch_flight(state, id)             # the FIRST caller's single flight
    {:noreply, put_in(state.flights[id], {ref, [from]})}
end

The flight is a spawn_monitord task, so the owner is never blocked: it GETs the L2 key, and on a miss runs the declared load_game/1 loader (the fall-through to Postgres), SETs both layers, and sends back one result. handle_info({:flight, id, result}) replies the one answer to every waiter and clears the flight; a flight crash fails all its waiters with {:flight_crashed, reason} — no caller wedges.

Two more guards keep the cache bounded and herd-free. Expiry is jittered so a cohort of games filled together never expires in step; and a full cache degrades to pass-through, it never fails.

# echo/apps/echo_store — the jittered clock + the full-cache degrade
# base = now + ttl; spread = ttl * jitter; the deadline is base ± a uniform band
# of width spread — so no two rows filled together share an exact expiry instant.
defp expires_at(spec) do
  base = System.monotonic_time(:millisecond) + spec.ttl_ms
  spread = trunc(spec.ttl_ms * spec.jitter)
  if spread == 0, do: base, else: base + :rand.uniform(2 * spread + 1) - spread - 1
end

# a full cache serves its caller from L2/loader and SKIPS only the cache write:
cond do
  size < state.spec.max_size -> :ets.insert(state.name, {id, value, expires_at(state.spec), version})
  reclaim(state) > 0         -> :ets.insert(state.name, {id, value, expires_at(state.spec), version})
  true ->
    :counters.add(state.spec.counters, counter(:full_skips), 1)
    :skip                       # degrade to pass-through, never refuse a read
end

The cache reports itself honestly through EchoStore.Table.stats(:cm_games) — the live counters (hits, misses, fills, l2_hits, coalesced, swept, full_skips, sweeps) plus the ETS size. The manuscript states the law plainly: “a cohort filled together never expires together and a herd never forms at the second boundary; a sweeper reclaims dead rows on a fixed tick, so memory is bounded by the declaration rather than by luck.”

One fill per herd, a jittered clock, and a full cache that passes through — the hot read stays cheap under load and the footprint stays what the table declared.

Move 4 — invalidate from another node

The games and emoji-set caches are coherence: :none because their entities are immutable for the game's life — they can never go stale. But codemojex's sessions table (:cm_sessions, kind SES) is the first mutable one, and there the third law bites: a write on one node must not leave a stale read on another. When a player's session is revoked, every BEAM node holding it in L1 must drop it — fast.

# echo/apps/codemojex — the sessions table is declared :tracking (cm.4 auth)
# RESP3 server-assisted client-side caching: Valkey itself pushes an invalidation
# for any write or DEL to the ecc:{cm_sessions}: prefix, and the owner evicts the
# L1 row — so a revoked SES is dropped from EVERY node's L1 immediately. :none here
# would be a defect: a revoked session surviving in L1 would keep authenticating.
{EchoStore.Table,
 name: :cm_sessions, kind: "SES", loader: &load_session/1,
 coherence: :tracking, ttl_ms: 86_400_000, max_size: 100_000}

For a versioned write that costs money if read stale, the same pillar carries coherence as a message about a name — EchoStore.Coherence. A write mints a new version, and an invalidation carries exactly two identities (the cached id and the writer's mint-time version, 29 bytes) over a broadcast lane (broadcast/4 → PUBLISH ecc:{table}:coh, at-most-once) or a job lane (enqueue/5 → Lanes.enqueue, at-least-once). Node B applies it with the conditional drop, which deletes the L2 row only if the incoming version is newer.

# echo/apps/echo_store — EchoStore.Coherence.drop_l2/4 (newer-wins, the named handle)
# Conditionally drop the L2 row: deleted only when `version` is newer than the
# version framed into the stored value — one transition, one script, so a late
# stale invalidation can NEVER erase a newer row. The one Lua of the pillar (the
# :coherence_drop script) lives in module 03 — linked, not re-printed here.
def drop_l2(conn, table, id, version) do
  Connector.eval(conn, @drop, [Keyspace.key(table, id)], [version])
end

newer?/2 is the whole engine: it compares the 11-byte snowflake payloads of two branded ids and ignores the namespace, so the order theorem's lexicographic == chronological property holds across kinds — coherence needs no coordinator, no lock, and no clock but the one already inside every id. Application is idempotent by construction (the same message applied twice answers stale the second time).

The lane that remembers

The deep durability — a crash-surviving coherence job whose applied table remembers the last version per name even after L1 forgot the row — folds to the durable journal floor at Echo Persistence. The full Lua body and the two lanes are taught in module 03; this move names the move.

One write becomes one small message about a name; every other node decides newer or not by the clock minted into that name — and a revoked session is gone everywhere at once.

The cache, built — one game's reads

Run a sequence of reads against a freshly declared :cm_games cache and watch it warm. The first read for a game misses in both layers and runs the loader; a concurrent burst on that game coalesces onto one flight; later reads are hits. A wrong-kind id is refused by the gate; an expiry sweep clears the row; a coherence message drops it newer-wins. The GAM ids are real branded ids decoded the same way the footer stamp is — the value's version is a mint-time id.

The read sequence · advance the cache

GAM idsourcewhat happened

The builder runs the real read path: Codemojex.Cache.fetch_game/1 over EchoStore.Table.fetch/3, with the kind gate at the door and the single-flight coalesce on the miss. Each row's GAM id is the game's own branded id — the same id that frames the cached value as its version, that the gate checks against the :cm_games kind, and that a coherence comparison would read as its clock. One id, three jobs.

One cache, three laws — declared so it exists, hit at ETS speed behind the gate, herd-proofed and bounded, kept coherent by the clock in every id.

Pattern → implementation

A read-through cache, single-flighted under a herd, with bus-coherent invalidation is the pattern the Redis Patterns Applied course frames — and its caching chapter applies these same moves to this same codemojex read surface from the pattern side. Here it is built from the implementation side.

the pattern (Redis Patterns Applied)

Cache-aside with a near-cache: serve a read from local memory, fill once per herd on a miss, jitter the TTL, and invalidate coherently when another node writes. R1 · Caching applies the cache-aside, stampede-prevention, and session patterns from that side.

the implementation (echo_store)

EchoStore.Directory.register/3, EchoStore.Table.fetch/3 (the single-flight + jittered clock), and EchoStore.Coherence.drop_l2/4 over the real Codemojex.{Cache, Tables, Store} seam — the branded GAM id the thread through all of them.

The same near-cache idea, read two ways: the pattern names the moves; the pillar shows the shipped code that makes them, end to end.

Recap — the pillar, exercised

The Cache pillar taught three laws over two layers; this workshop built one codemojex near-cache out of all three. Declare the cache so it exists, hit at ETS speed with the kind gate at the door, survive a herd with one fill and a jittered clock, invalidate from another node with a message about a name. The games cache never goes stale (:none); the sessions cache is evicted everywhere the instant it is revoked (:tracking). The thread through all four is one branded id — the entity's own 14-byte id is the value's version, the coherence clock, and the key the gate checks, minted once and carried unchanged.

The next floor down — the lane that remembers a coherence verdict past a crash, the durability dial in full — is the persistence floor. And the architecture law these figures realize is the BCS store chapter.

References

Sources

Erlang/OTP — the ets module — the public, read-concurrent L1 table a hit reads directly, in the caller's process.
Valkey — Cluster specification — the {cm_games} hashtag landing every ecc: key of one cache on one of 16384 slots.
Valkey — SET — the SET … PX the writer and the flight issue direct to L2.
Söderqvist — A new hash table — the L2 Valkey the near-cache fronts, costed at rest.
Helland — Life Beyond Distributed Transactions — the entity addressed by a key, cached close to where it is used.
King — Announcing Snowflake — the time-ordered id every cached value is framed with, whose byte order is the coherence comparison.

Related in this course

Cache-aside, two layers — moves 1–2: the declared tiers and the cache-aside read.
Single-flight & jittered TTL — move 3: one fill per herd, the jittered clock, the degrade.
Coherence on the Bus — move 4: a message about a name, the two lanes, the conditional drop.
The Cache — the pillar landing this workshop closes.
The Queue — the fair lanes a crash-surviving coherence job rides.
The Bus — the wire a coherence broadcast rides.
redis-patterns · Caching — the pattern side of the door.
bcs · the Store — the manuscript chapter (B4) these figures realize.
The persistence floor — the durable journal that remembers a coherence verdict past a crash.