jonnify·persistence /echo-persistence/platform/bus-and-persistence

Module 13 · Chapter IV — The platform

EchoBus + Echo Persistence.

Eleven modules built two things that look separate: a message bus (EchoMQ) and a durable floor (the engines and store of EP1–10). This module is the claim that they are one system. They meet at exactly three seams — a shared cursor, an outbox, and a fold — and through those seams a write becomes a loop across four tiers: accepted, committed, published, and folded back. Tap each seam below to see what crosses it.

Two halves, three seams. The bus (left) and the durable floor (right) are coupled in both directions. ① the commit LSN published from the engine is the cursor the bus binds to; ② the outbox stands beside the bus, journaling intents while the bus stays on Valkey; ③ the archive fold commits stream slices back into the engine.
EchoMQ — the bus accept tier · Champ (fsync / K) stream tier · emq3.0 streams fast, reliable — but volatile EP11 / the Stream Tier the durable floor · EP1–10 engine · Graft (commit, LSN) store · CubDB → Tigris per-commit, replicated, recoverable EP7–10 / the engines ① commit LSN = cursor engine publishes the LSN → bus binds replicas ② outbox beside the bus bus stays Valkey · low-volume intents journaled ③ archive fold (emq3.5) stream slice → EchoStore.Graft.commit
readout · the bus and the durable floor are coupled both ways. Tap ①, ②, ③ to follow what crosses each seam — each is a dive below.
§1

Why it is one system, not two

graft.design.md

It is tempting to read the course as two stacks bolted together: a queue on one side, a database on the other. The platform’s design refuses that split, and the reason is the commit LSN. When the accept tier (Champ, in-memory with an fsync every K records) folds a batch into a Graft transaction, that batch becomes one LSN replicated to Tigris — and that same LSN is published over EchoMQ. A number that means “this much is durable” on the store side is the identical number that means “subscribe from here” on the bus side. So the two halves do not exchange data through an adapter that could drift; they share a cursor. That single fact is what makes a replica’s position, a consumer’s offset, and a commit’s durability the same coordinate, and it is why the rest of the module can talk about a loop rather than a pipeline: the engine’s commit drives the bus, and — through the archive fold — the bus drives the engine’s commit. Everything else in this module is a consequence of choosing one cursor for both worlds.

§2

The three seams

graft.engine-split.design.md · emq3.5

The coupling is not diffuse; it is exactly three seams, and each is a dive. The cursor (①) is the accept–commit join: the caller picks the guarantee per call — :async returns once the batch is fsync’d locally, :sync only after the Tigris-replicated commit acknowledges — and the published LSN binds every replica to the same point. The outbox (②) is the deliberate non-coupling of the hot path: the bus stays on Valkey because it must be fast, and durability stands beside it as a low-volume transactional-enqueue outbox (EchoStore.Durability), so a durable write never enters the enqueue path — its adapters are SQLite and Memory, with the Graft commit-log as a bring-your-own option. The fold (③) is the return leg: the Stream Tier’s retention does not delete history, it archives it — a dedicated fold consumer commits trimmed stream slices into the engine with EchoStore.Graft.commit/3, fold-before-trim, so the bus’s old tail becomes the store’s durable segments. Read the seams in order and the loop is complete: a write is accepted, committed (①), recorded beside the bus (②), carried as events, and folded back into the store (③).

§3

The three dives

module 13
§4

Up next

chapter IV
§5

References & sources