jonnify·persistence /echo-persistence

Echo Persistence · 13 modules · 4 chapters

Durable state, taught as built.

EchoMQ enqueues on Valkey — fast and volatile. Echo Persistence is the floor beneath it: an in-heap accept tier, a transactional page-store built twice (Elixir on CubDB, Rust on Fjall), and replication to Tigris. The course builds that ladder from the bottom up, and every module is an interactive dive into a mechanism, not a table of machine numbers.

Start the overview Module 1 →
The storage ladder. The shape the course builds. Tap a tier for its job; tap a path to see how a read and a commit move through it.
ETSin-process L1 head cache Valkeythe bus (RESP3) + the L2 CubDB / Fjalldurable local page tier · the LSN log Tigrisremote object storage · off-box read · fault up commit · replicate down
readout · Tap a tier to read its job, or a path to trace a read (faults up the ladder) or a commit (appends local, replicates down to Tigris).
§1

The course map

13 modules · 3 dives each
built ready to open planned specified, building soon on the runway
Ch. I

Foundations

Open chapter →The mechanism and the model — why durability is one knob, and the two-tier shape that turns it.
Module 1built

The durability spectrum

The one knob behind every tier — durable records per fsync — and the two axes a queue is placed on.
Module 2built

Champ · the accept tier

In-heap accept with a checkpoint per K; recovery and replay as mechanism, not a number.
Module 3built

Persistence concepts

Volume, LSN, snapshot, the OCC commit, and the conditional-write fence — engine-neutral.
Ch. II

The local store

Open chapter →CubDB in depth — the append-only immutable B-tree under the native engine, and what it makes possible.
Module 4built

CubDB architecture

Append-only immutable B-tree; how the page log is built on it; its place among the tiers.
Module 5built

MVCC & time travel

Zero-cost snapshots from immutability; reading a Volume at any past LSN, lock-free.
Module 6built

Replay & recovery

The log as truth; green boot, replica catch-up, and feed replay as one idea.
Ch. III

The engines

Open chapter →Built twice and bridged — the native Elixir engine, the Rust twin, replication, and the cross-runtime contract.
Module 7built

The native Elixir engine

EchoStore.Graft.* on CubDB — the write-lock, OCC, the streamer, the lazy reader.
Module 8built

The Rust engine

echo_graft on Fjall + OpenDAL — the fork, the runtime, the in-memory feed.
Module 9built

Tigris & the fence

Push rollup, the create-if-not-exists fence, and log-head recovery — the shared remote.
Module 10built

The BEAM↔Rust contract

eg.4 — the byte-frozen wire, the backend, and the compositional cross-runtime proof.
Module 11planned

Postgres WAL on the floor

PROPOSED — the consistency-first ledger’s WAL LSN onto the Graft+Tigris floor and the one cursor; the fork ruled A→B.
Ch. IV

The platform

Open chapter →The whole picture — the bus itself, persistence beneath it, why it beats classical scheduling, and the door it opens onto BCS.
Module 12built

EchoMQ Bus

ValKey Streams internals and the EchoMQ Stream Tier built on them — the verbs, the PEL, retention.
Module 13built

EchoBus + Echo Persistence

The composition — EchoMQ beside the durable floor of EP1–10; the loop across four tiers, the archive fold.
Module 14built

Why it beats classical scheduling

Postgres-backed Oban, honestly — what Echo separates where Oban couples, and where each one wins.
Module 15built

The door to BCS with Echo Persistence

The durable log plus Tables & Properties as a substrate for building systems — worked on Codemoji.
§2

References & sources