jonnify·persistence /echo-persistence/durability-spectrum/the-shootout-and-the-knob

Module 1 · Dive 1.2

The shootout and the one knob.

A four-way measurement once spanned three orders of magnitude, and people remember the digits. The digits are a distraction. The whole spread comes from one lever: how many durable records ride each fsync. Turn it below and watch throughput follow — the four engines are just stops on this dial.

§1

The one knob

interactive model
fsync amortization. The same N records, regrouped. Raise records-per-fsync: fewer flushes carry the same work, and the throughput bar climbs — until per-record cost caps it.
N = 24 records · grouped by records-per-fsync fsyncs: 24 throughput (relative to commit-per-record) ×1.0
readout · At one record per fsync, every write pays a full flush — the strict, slow corner. Slide right: each flush now carries many records, so throughput rises roughly in proportion, then flattens as the per-record work dominates.

That flattening is the whole story of the spectrum. Early on, throughput climbs almost linearly with records-per-fsync, because the flush is the dominant cost and you are dividing it across more records. Past a point, the per-record work (serialize, append) dominates and more batching buys little. Every engine lives somewhere on this curve; the only question is what sets its records-per-fsync.

§2

The engines as stops on the dial

graft.design.md

Read the four-way result as positions, not numbers. Memory has no flush, so it sits at the far ceiling. Oban’s single insert is one record per flush — the strict, slow end; its insert_all puts a transaction’s worth of records behind one flush and jumps up the curve. BullMQ’s bulk add does the same on Redis. Champ never makes the caller hold a batch at all: its checkpoint-every-K keeps records-per-fsync high continuously, so its single-record path already sits where the others only reach with a bulk API.

enginestoredurabilityenqueue (single)enqueue (bulk)
Memoryin-memory mapnone336,961n/a — no fsync to amortize
Champ K=1,000local fsync snapshotbounded ≤ 1,000 rec33,989n/a — K amortizes continuously
Champ K=10,000local fsync snapshotbounded ≤ 10,000 rec103,970n/a — K amortizes continuously
BullMQRedis AOF everysecbounded 1s6,38816,180
ObanPostgres sync commitstrict per-commit71813,235
Champ + Graftaccept tier + commit → Tigrisstrict, replicatedpending eg.6 · per-workloadpending eg.6

Reading the table. Figures are indicative single-core measurements against MinIO — positions on the dial above, not a portable benchmark. “Bulk” is durable-enqueue via a bulk write API (insert_all / addBulk), amortizing the fsync on the write side; n/a means the row needs no such API (Memory has no fsync; Champ already amortizes via K). EchoMQ’s shipped batches (emq.5 claim_batch/BatchConsumer) are claim/consume batches — a delivery axis, durability-neutral — not this column. Champ + Graft is the two-tier design; its number is measured per-workload at eg.6, not asserted here.

§3

References & sources