B4.4 · dive 3 — coverage and the price
The outbox empties, the memory does not.
The same predicate that drives replay retires the outbox: an intent is deletable when its
name carries an applied version at least as new — coverage, not acknowledgment. The H5 gate runs
it over H4's fifty intents; the H6 gate prices the whole arrangement and closes the module:
143 us per record-and-mark pair at the writer's edge, and a remembered lane end-to-end median of
524 us against the bare lane's committed 148 us -- 3.5 times the latency buys an outbox, a last word per
name, and a replay that survives the bus.
Source: content/bcs4.4.md, quoting bcs_rung_4_4_check.out. This dive
reads the compaction derive, H5, the price derive, H6, and the record's close — plus the fix the gate
forced on the way to 143: the first measurement was 224.
Static walk — 1) after H4, all 50 intents are deletable: their names carry applied versions at least as new · 2) compact retires all 50 in one pass · 3) the applied memory keeps its 50 names · 4) replay over the compacted journal finds nothing to do · 5) a fresh open of the same file still answers the last word.
The transcript
bcs_rung_4_4_check.out · verbatim · this dive reads the compaction derive, H5, the price derive, H6… (the self-check, the header, and H1–H4 open the record -- frozen on the module hub) derive (compaction): an intent is retired when its name carries an applied version at least as new -- coverage, not acknowledgment -- so after H4 all 50 intents are deletable, the applied memory keeps all 50 names, replay finds nothing, and a reopen still remembers H5 compaction ok -- all 50 intents retired by coverage in one pass, the applied memory kept its 50 names, replay over the compacted journal found nothing to do, and a fresh open of the same file still answers the last word -- the outbox empties, the memory does not derive (price): on prepared-once statements -- the single writer's privilege, with bind resetting the statement -- the writer's pair is two WAL commits and one cached rowid read at synchronous=NORMAL, so expect between 20 and 250 us on this disk; the remembered lane's end-to-end median should land between 200 us and 2 ms against the bare lane's committed 148 us: dearer, bounded, and the chapter's reason the journal is declared per group rather than assumed H6 price ok -- the memory's price on this disk: 143 us per record-and-mark pair at the writer's edge, and a remembered lane end-to-end median of 524 us against the bare lane's committed 148 us -- 3.5 times the latency buys an outbox, a last word per name, and a replay that survives the bus PASS 6/6
The price derive is the chapter's argument in one line: the band is derived from the WAL's commit shape before the stopwatch runs, the measurement lands inside it, and the closing clause — "dearer, bounded, and the chapter's reason the journal is declared per group rather than assumed" — is the declaration discipline of this whole part, applied to durability.
Compaction is coverage
content/bcs4.4.md · What · Decisions · WhenCoverage, not acknowledgment. An intent is retired when its name carries an applied version at least as new. The hot path pays no per-intent completion write; replay and compaction share one predicate; and the applied table does double duty as the dedup floor and the retention rule. After H4's drill all 50 intents are covered, so one pass retires them all — and the gate then proves the two halves part ways: replay over the compacted journal found nothing to do, and a fresh open of the same file still answers the last word.
{:ok, retired} = Journal.compact(:limits_journal)
Run compact on a timer sized to the replay-window appetite: intents are
small, coverage is one SQL pass, and an outbox kept under a few thousand rows replays in milliseconds.
Both verbs are idempotent and safe to run on a schedule — replay/2 the recovery verb,
compact/1 the retention verb.
The price, and how the gate improved the code
content/bcs4.4.md · What · Where · BoundariesThe first measurement of the writer's pair came in at 224 microseconds against a
derivation that priced WAL commits — the gap was per-call statement preparation, "three SQL parses
per pair". The fix is the single writer's privilege: the five hot statements are prepared once at
journal start and rebound per call (exqlite's bind resets the statement), and the pair fell to the
committed 143 us per record-and-mark pair. At synchronous=NORMAL the WAL issues
no per-commit sync — "the checkpoint is the only operation to issue an I/O barrier" — so the
pair is two log appends and a cached rowid read.
Declare a journal per group when losing that group's queued coherence to a bus restart
costs more than 143 microseconds per write ever will — and skip it for surfaces where the TTL floor was
always acceptable; the broadcast lane, in particular, gains nothing from an outbox it would only drop. The
toolchain fact behind the header: the runtime's mix project grew its first compiled dependency —
exqlite and its build chain vendored as path deps under
runtimes/elixir/vendor/, tarballs from the package mirror, the NIF compiled from the bundled
SQLite amalgamation, no system SQLite consulted — and the rung runs under MIX_ENV=prod mix
run, the first cache rung to do so, with the plain-script tower untouched.
Boundaries, stated honestly: synchronous=NORMAL is a stated trade — every
process crash, consumer kill, and bus restart in this part is fully covered, and a machine power loss may
trim the unsynced tail of the WAL. The layer that closes that gap is "Litestream-shaped …
referenced and deliberately not implemented here": a separate process streaming the journal off the
box, kept outside this codebase by the same separation D-2 already taught. The journal is per-group and
per-node; replay is at-least-once by construction and harmless by comparison. And the microseconds carry
their header — one scheduler, this container's disk, a 29-byte payload; "the 3.5 multiple travels
better than the 143." The manuscript plans the measured comparison set as B4.5 · The
Cache Referee; until that chapter ships, no comparative figure exists. The applier under the lane
is the sibling module's engine — B4.3 · The Single Writer and the Ring.
References
sources · relatedSources
- Richardson — Pattern: Transactional outboxthe pattern whose price this gate states
- SQLite — Write-Ahead Loggingthe commit path and the NORMAL trade — the checkpoint as the sole sync barrier
- Litestream — How it worksthe off-box durability layer, named and deliberately not implemented
Related
- B4.4 — The Lane That Remembersthe module hub; the full rung in context
- B4 — EchoStorethe chapter landing; Part IV's arc
- B4.2 — Coherence by Mint Timethe bare lane's committed 148 us
- B4.1 — Cache-Aside at ETS Speedthe declared table the journal stands behind
- B3 — The Busthe volatile substrate D-2 keeps
- EchoMQ — the coursethe protocol in rung-level depth
- Redis Patterns Appliedthe substrate under the bus
- Functional Programming in Elixirthe umbrella the runtimes live in