jonnify·persistence /echo-persistence/engines/tigris+fence

Module 9 · Chapter III — The engines

Tigris & the fence.

Both engines, however different inside, write the same remote: a Tigris bucket holding two kinds of object — page segments and the commit chain that is the durable log head. Three operations run against it: push rollup coalesces changed pages into a segment, the create-if-not-exists fence picks one winner per commit, and log-head recovery reads the head back. Tap an operation to see which objects it touches.

The shared remote. Both engines PUT into one Tigris bucket of segments and commit objects, with a head pointer at the tip. Tap an operation to light the objects and paths it uses.
native · Elixir CubDB · direct Tigris Rust Fjall · OpenDAL Tigris · object storage (the shared remote) commit/ — the durable log head head → segments/ — page data PUT cold node reads head on recovery
readout · Tap an operation. Both engines PUT into one bucket: segments hold page data, the commit chain is the durable log head. The remote is shared storage and, through the fence, a shared coordination point.
§1

One remote, two engines, three jobs

graft.roadmap.md · eg.2

The remote is where the two engines stop being different. Whatever a Volume is locally — a CubDB keyspace or a Fjall partition — its durable, shareable form is the same: Zstd-framed page segments and a chain of commit objects whose tip is the log head, all in one Tigris bucket reached over OpenDAL (S3-compatible, SigV4). Three operations act on it. Push rollup turns the pages changed since the last upload into one segment. The fence — a conditional create-if-not-exists on the next commit object — lets the bucket itself decide which writer owns each commit, so even two engines or a promoting replica can never fork the log; in the Rust engine this is put_commit with if_not_exists returning ConditionNotMatch on a loss, and in the native engine the matching create-only If-None-Match PUT — the same protocol, two implementations, which is exactly what rung eg.2 verifies. And log-head recovery lets any cold node read the head back from the bucket and start serving, faulting pages on demand. Each is a dive.

§2

The three dives

module 9
Dive 9.1

Push rollup

Many page writes, some repeated, become one segment: the rollup keeps the latest version per page, frames it with Zstd, and uploads a single object. Watch the writes collapse.

 Dive 9.2

The create-if-not-exists fence

Three different writers race for one commit slot; the bucket grants exactly one. Watch the losers step down and the commit chain stay strictly linear — cross-engine, no fork.

 Dive 9.3

Log-head recovery

A cold node with nothing local reads the head and commit chain from the bucket, then faults pages from segments on demand — no full download. Step a recovery through.
§3

Build & check

What you build

Name the two object kinds in the bucket and what each is for, then map the three operations onto them: which writes segments, which writes a commit object, which reads them back. The bucket layout is the deliverable.

Check

Answer without code: what single object-store primitive lets the remote arbitrate commits across both engines, and why can’t the log fork? “Create-if-not-exists on the commit object” means you have the module.

§4

References & sources