jonnify·persistence /echo-persistence/overview/persistence-in-the-platform

EP · Overview · Dive 3

Where persistence sits.

The platform is two halves that compose: Echo Bus (EchoMQ on Valkey) carries the work, and Echo Persistence keeps the state. Persistence spans four storage tiers, and a queue is placed on a durability dial across them — the only durability the enqueue hot path touches is a small, low-volume outbox beside the bus.

§1

The four storage tiers

graft.engine-split.design.md

State moves down a ladder of tiers, each with a different job. ETS is the in-process L1 head cache; Valkey is the bus and the L2; an embedded immutable store — CubDB for the native engine, Fjall for the Rust engine — is the durable local page tier; Tigris is the remote, off-box floor. A read faults up the ladder; a commit appends locally and replicates down to Tigris.

echo · the storage tiers, top to bottomAS-BUILT
ETS            in-process L1 head cache        derived, safe to drop
Valkey         the bus (RESP3) + the L2        EchoMQ, fast, volatile
CubDB / Fjall  durable local page tier         append-only, MVCC, the LSN log
Tigris         remote object storage           /segments + /logs/.../commits/{LSN}
read → fault up the ladder · commit → append local, replicate down to Tigris

The page tier itself is built twice and the two engines coexist by ruling: the native Elixir EchoStore.Graft.* on CubDB is the canonical default; the Rust echo_graft on Fjall + OpenDAL serves raw-page and replica-recovery workloads, driven over the bus under a non-colliding name. Both replicate to Tigris behind the same conditional-write fence.

§2

The durability dial

graft.design.md · durability.ex

Each queue is placed on a dial: Memory (holding nothing), Champ-K (a bounded window), or Champ + a page engine + Tigris (per-commit and replicated). Crucially, the only durability the enqueue hot path touches is the outbox — a low-volume intent journal standing beside the bus, a small mostly-idle dependency, not the path every dequeue, heartbeat, and ack runs through. The bus stays on Valkey; the page tier serves the page/replica need separately.

The dial. Select a queue to read which tiers it engages.
readout · Select a queue. The bus carries every queue; persistence engages by tier.

EchoMQ itself dogfoods the page engine as its durable spine, and the planned Stream Tier archives there — so Echo Persistence is not a bolt-on beneath the bus, it is the floor the bus stands on. The thirteen course modules build this ladder from the bottom up.

§3

References & sources