jonnifyechomq

Cache-aside, two layers · dive 03

Writing
both layers

Every write frames the value with its mint-time version — L2 first, then L1. The version is the seed of coherence.

A write to EchoStore.Table always goes to both layers, in order. put/3 mints a version of the table’s kind — the write is its own event, identified by its own 14-byte branded id. put/4 carries the caller’s own version when the writer already has the id it wants to pin. The L2 value is the frame version <> value: a 14-byte branded id prepended to the binary payload, so a future read can always recover both from a single GET. invalidate/3 is the admin verb — it drops both layers unconditionally, with no version check. The version-guarded drop, newer-wins, belongs to module 03.

put/3 and put/4

put/3 mints a fresh version: EchoData.BrandedId.generate!(spec.kind) at the moment of the write. put/4 takes an explicit <<_::binary-14>> = version from the caller.

Both arities gate on the kind (wrong namespace → {:error, :kind}) and then delegate to the owner’s handle_call({:put, ...}).

# EchoStore.Table — handle_call({:put, id, value, version}, ...)
# Both layers, in order: L2 first, then L1.
# Reached from both put/3 (minted version) and put/4 (caller's version).
def handle_call({:put, id, value, version}, _from, state) do
  # L2 first: compose the Valkey key and SET the framed value.
  # The frame is version <> value — the 14-byte branded id prepended
  # to the binary so a future GET can split it with <>.
  # PX ttl_ms: key expires in ttl_ms milliseconds.
  l2 = Keyspace.key(state.table, id)

  {:ok, "OK"} =
    Connector.command(state.conn, [
      "SET",
      l2,
      version <> value,   # the frame: version prepended to the payload
      "PX",
      Integer.to_string(state.spec.ttl_ms)
    ])

  # L1 second: insert the ETS row {id, value, expires_at, version}.
  # expires_at is jittered: ttl ± ttl*jitter (monotonic ms).
  # The version is stored in the row tuple so a coherence comparison
  # can read it without decoding the value binary.
  insert(state, id, value, version)
  {:reply, :ok, state}
end

The outer put/3 mints the version immediately before delegating:

# EchoStore.Table — put/3 (public)
# Mints a version of the table's kind at the moment of the write.
# BrandedId.generate!(spec.kind) mints a 14-byte branded id, e.g. "RMM0Kp7..."
# The write is its own event, identified by its own snowflake.
def put(name, id, value) when is_binary(value) do
  case EchoStore.spec(name) do
    :error -> {:error, :no_such_cache}
    {:ok, spec} -> put(name, id, value, EchoData.BrandedId.generate!(spec.kind))
  end
end

L2 first, L1 second — in that order, every time. L2 is the durable shared tier; L1 is the fast local tier. A reader that misses L1 but hits L2 always finds a framed value it can split.

The L2 frame: version <> value

Every value stored in Valkey is the binary concatenation of two parts: the version (a 14-byte branded id, minted at write time) and the value (the application binary). The frame is split on read:

# The L2 frame — how a flight splits a stored value
# When GET ecc:{table}:id returns a binary, the flight matches it:
{:ok, <<version::binary-14, value::binary>>} ->
  {:l2, value, version}   # both parts recovered in one match, no extra GET

# The "short" guard catches a malformed frame (should never happen in practice):
{:ok, _short} ->
  {:error, :corrupt_l2_frame}

The version is always 14 bytes (a branded id is always 3 namespace bytes + 11 Base62 characters). The value can be any binary length, including zero. The pattern match <<version::binary-14, value::binary>> is always unambiguous.

The version stored in the frame is also stored in the L1 row tuple as the fourth element {id, value, expires_at, version}. Module 03 uses it for the newer-wins comparison without touching the value binary.

The frame is the contract: wherever a value goes — L2, L1, a coherence message, a fill reply — its version travels with it. The version is not metadata. It is part of the value.

invalidate/3 — the admin verb

invalidate/3 drops both layers unconditionally. It is the operator’s escape hatch — used when correctness requires eviction regardless of which version is current. It is distinct from the coherence drop (module 03), which guards on the version and only evicts if the incoming version is newer than the row’s.

# EchoStore.Table — handle_call({:invalidate, id}, ...)
# The admin verb: DEL L2, :ets.delete L1, unconditionally.
# No version check. This is the operator path: use when you need the
# value evicted regardless of what version is current.
# Distinct from the coherence drop (module 03), which is version-guarded.
def handle_call({:invalidate, id}, _from, state) do
  l2 = Keyspace.key(state.table, id)
  {:ok, _} = Connector.command(state.conn, ["DEL", l2])  # drop L2
  :ets.delete(state.name, id)                             # drop L1
  {:reply, :ok, state}
end

Two drops, no guard. The admin verb is a sledgehammer — the right tool when you need the entry gone and do not care about the race. The coherence drop is the scalpel.

Write simulator — frame and recover

Pick an operation. The simulator frames a write (composing version <> value), reads back the frame (splitting it at byte 14), and shows both the L2 Valkey command and the L1 row that would be inserted. All pure JS, no network. The version ids are illustrative; the 14-byte structure and the binary match are real.

Write operations · frame, recover, drop

The L2 frame: version <> value

version binary-14 bytes e.g. RMM0Kp7qW2Nmnx value binary (any length) e.g. "room:lounge" ... byte 14 <<version::binary-14, value::binary>> — one pattern match, both parts

The frame is a binary contract: 14 bytes of identity, then arbitrarily many bytes of payload. The version is always recoverable from a single GET, with no separate metadata key and no schema.

Pattern & implementation

The pattern

Cache-aside write. The application writes to the source of truth and to the cache in the same call. The cache value is stamped with the write’s identity so a later reader can detect staleness. The cache is never authoritative — the source of truth is — but the stamp makes comparison possible without asking the authority again.

EchoStore implementation

put/3 mints the stamp (BrandedId.generate!(kind)); put/4 carries the caller’s. The write goes to L2 (SET ecc:{table}:id (version <> value) PX ttl_ms) then L1 (insert/4 → {id, value, expires_at, version}). invalidate/3 is the unconditional eviction; the version-guarded eviction is module 03’s coherence drop.

The version is not an afterthought. It is written with the value, always, so that every future reader — a fill, a coherence message, a newer-wins comparison — has the information it needs without a second round-trip.

References

Sources

Related in this course