jonnify·persistence /echo-persistence/engines/native-elixir

Module 7 · Chapter III — The engines

The native Elixir engine.

Everything so far has been mechanism. Here it becomes a running engine: EchoStore.Graft, pure Elixir on CubDB, the platform’s default. A handful of OTP processes implement the ideas you already know — a VolumeServer serializes commits, a lazy Reader faults through the tiers, a Committer drains the commit-log-as-outbox to object storage. Tap a process to trace its path.

The engine, as OTP processes. Solid boxes are processes, dashed are stores. Tap a process to light its data path — each is a dive.
clients VolumeServer write-lock · OCC · VOL… Reader lazy fault · L1 → disk → remote commit log = outbox · CMT… CubDB durable pages · SEG… ETS L1 head cache Committer drain outbox · fence upload Tigris segments · conditional PUT
readout · Tap a process. The VolumeServer takes commits and writes pages; the Reader serves reads by faulting through the tiers; the Committer ships the commit log to Tigris. Three processes, the whole engine.
§1

Ideas, now processes

graft.engine-split.design.md

The native engine is deliberately small because Chapters I–II did the hard thinking. The VolumeServer is a GenServer whose single mailbox gives the strict commit order Module 3 needed, doing the OCC head-check and appending pages to CubDB. The Reader is the lazy read path: the head cache in ETS, then CubDB’s pages, then a segment fetched from Tigris on a miss. The Committer treats the commit log as an outbox, draining it to object storage behind the conditional-write fence — the same log that is durability locally is the replication source remotely. Branded ids tie it together: VOL for volumes, SEG for segments, CMT for commits.

§2

The three dives

module 7
Dive 7.1

The VolumeServer

The write path. One GenServer mailbox serializes commits — the OCC check needs no lock because the mailbox already is one. Queue commits and watch them apply in order.

 Dive 7.2

The lazy Reader

The read path. A page request faults through ETS, then CubDB, then a remote segment — and the fetched page is cached on the way back. Read pages and watch the fault.

 Dive 7.3

The commit-log outbox

One log, two jobs: durable locally, pending upload remotely. Watch the Committer drain it to Tigris behind the fence, marking each segment shipped.
§3

Build & check

What you build

Name each process and the one chapter idea it implements: VolumeServer → (Module 3’s …), Reader → (the storage ladder’s …), Committer → (Module 6’s …). If you can fill the blanks, the engine is just the foundations wired together.

Check

Answer without code: why does the VolumeServer need no explicit lock, and what are the two roles of the commit log? “The mailbox serializes” and “durability + outbox” mean you have the module.

§4

References & sources