Module 1 · Dive 1.2
The shootout and the one knob.
A four-way measurement once spanned three orders of magnitude, and people remember the digits. The digits are a distraction. The whole spread comes from one lever: how many durable records ride each fsync. Turn it below and watch throughput follow — the four engines are just stops on this dial.
The one knob
interactive modelThat flattening is the whole story of the spectrum. Early on, throughput climbs almost linearly with records-per-fsync, because the flush is the dominant cost and you are dividing it across more records. Past a point, the per-record work (serialize, append) dominates and more batching buys little. Every engine lives somewhere on this curve; the only question is what sets its records-per-fsync.
The engines as stops on the dial
graft.design.mdRead the four-way result as positions, not numbers. Memory has no flush, so it sits at the far ceiling. Oban’s single insert is one record per flush — the strict, slow end; its insert_all puts a transaction’s worth of records behind one flush and jumps up the curve. BullMQ’s bulk add does the same on Redis. Champ never makes the caller hold a batch at all: its checkpoint-every-K keeps records-per-fsync high continuously, so its single-record path already sits where the others only reach with a bulk API.
| engine | store | durability | enqueue (single) | enqueue (bulk) |
|---|---|---|---|---|
| Memory | in-memory map | none | 336,961 | n/a — no fsync to amortize |
| Champ K=1,000 | local fsync snapshot | bounded ≤ 1,000 rec | 33,989 | n/a — K amortizes continuously |
| Champ K=10,000 | local fsync snapshot | bounded ≤ 10,000 rec | 103,970 | n/a — K amortizes continuously |
| BullMQ | Redis AOF everysec | bounded 1s | 6,388 | 16,180 |
| Oban | Postgres sync commit | strict per-commit | 718 | 13,235 |
| Champ + Graft | accept tier + commit → Tigris | strict, replicated | pending eg.6 · per-workload | pending eg.6 |
Reading the table. Figures are indicative single-core measurements against MinIO — positions on the dial above, not a portable benchmark. “Bulk” is durable-enqueue via a bulk write API (insert_all / addBulk), amortizing the fsync on the write side; n/a means the row needs no such API (Memory has no fsync; Champ already amortizes via K). EchoMQ’s shipped batches (emq.5 claim_batch/BatchConsumer) are claim/consume batches — a delivery axis, durability-neutral — not this column. Champ + Graft is the two-tier design; its number is measured per-workload at eg.6, not asserted here.
References & sources
- Echo records
- graft.design.mdthe shootout and the fsync reading
- emq.roadmap.mdemq.5 claim/consume batches — the other axis
- External
- oban-bg/obansingle insert vs insert_all
- taskforcesh/bullmqadd vs addBulk on Redis
- Designing Data-Intensive Applications, Kleppmann 2017group commit, the amortization idea