R7.01.3 · dive C
Persistence: RDB vs AOF.
Two durability mechanisms, one question: after a crash, how much do you lose? RDB snapshots via
fork(); AOF logs every write, and appendfsync sets the bound.
RDB writes a point-in-time snapshot — compact, fast restart, but everything since the last
snapshot is gone. AOF writes an append-only log; everysec bounds worst-case loss to about one second.
EchoMQ runs AOF-only (save "") with the RDB preamble — one fork source, a ~1s bound its
checkpoints assume. Select a mechanism; the readout reports the loss window.
RDB — a snapshot via fork()
redis-as-primary-database · RDBRDB takes a point-in-time snapshot: the server forks, the child writes the whole dataset to one compact file, and the parent keeps serving. The file is small and restarts from it are fast, which makes RDB a good fit for disaster-recovery backups, fast instance bootstrapping, and development environments. The cost is the window: a snapshot taken every K seconds means a crash loses up to K seconds of writes — the whole interval since the last snapshot, not a bounded slice of it.
# RDB: snapshot on a write-count rule (from the author source) save 900 1 # snapshot if 1 key changed in 900s save 300 10 # or 10 in 300s save 60 10000 # or 10000 in 60s — the tighter the rule, the smaller the window and the more forks
RDB alone is a backup mechanism, not a durability guarantee. The snapshot is the dataset as of the last fork; the writes after it live only in memory until the next one. For a record of truth that is not enough, which is why a primary store reaches for the log.
RDB is a compact, fork-written snapshot — fast to restart from, but a crash loses every write since the last snapshot: the whole snapshot interval, not a bounded slice.
AOF — the log, and the fsync policy
redis-as-primary-database · AOFAOF writes a log of every operation that changes the dataset. On restart the server replays the log and the
dataset is rebuilt. The loss bound is set by appendfsync — how often the log is flushed to the
disk. Three policies trade durability against latency:
policy loss bound latency use case always ~0 (maximum) high financial transactions everysec ~1 second low most production workloads no OS-dependent minimal a caching layer
always fsyncs on every write — no loss, but the disk is on the critical path of every command.
no hands the decision to the operating system — fast, but the loss bound is whatever the
kernel's flush interval happens to be. everysec flushes once a second on a background thread: the
worst case is about one second of writes, paid off the command path. For a primary store, everysec is
the standard choice.
Hybrid persistence sets aof-use-rdb-preamble yes: the AOF file starts with a compact RDB-format
snapshot, then the incremental commands since. The rewrite is small and the restart is fast, while the log still
bounds loss to the appendfsync window. It is not RDB snapshotting on a timer — it is the AOF
rewrite borrowing the RDB encoding.
AOF replays a log of every write; appendfsync picks the loss bound —
everysec trades ~1 second of worst-case loss for keeping the disk off the command path.
EchoMQ — AOF-only, one fork source
infra/valkey/conf/valkey.conf · persistenceThe textbook says enable both. EchoMQ's real infra/valkey/conf/valkey.conf does the opposite, and on
purpose. The persistence block is the as-built tuning surface — save "" turns RDB snapshotting
off, and the append-only log is the single source of durability.
# ---- Persistence: AOF is the single source of durability --------------------- # One fork source. AOF everysec bounds worst-case loss to about one second and # fsyncs on a background thread. RDB snapshotting is disabled so it does not # compete for a second fork; the AOF still uses an RDB preamble for fast rewrite. appendonly yes appendfsync everysec auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-use-rdb-preamble yes save ""
appendonly yes + appendfsync everysec make the append-only log the single source of
durability, with the ~1s bound paid on a background thread. aof-use-rdb-preamble yes keeps the
compact RDB format for the AOF rewrite, so a restart is still fast. save "" disables RDB
snapshotting. The reason both are not on is the fork: command execution is single-threaded, and a fork's
copy-on-write spike must be budgeted under the machine's RAM. Two independent fork sources — an RDB snapshot
timer and an AOF rewrite — can collide. Disabling RDB leaves one fork source, the AOF
rewrite, scheduled by auto-aof-rewrite-percentage.
“With
— The Branded Component System, B8.2 · Valkey on a Fly machineappendonlyon andappendfsync everysec, the volume holds an append-only log and the loss bound after a crash is roughly one second of writes, not a whole snapshot interval — and EchoMQ’s checkpoints are designed against exactly that bound.”
Notes on Valkeyappendfsync everysec flushes the append-only file
once a second on a background thread, so a crash loses at most about one second of writes; always
fsyncs every write and no defers to the OS —
valkey.io/topics/persistence.
RDB vs AOF — pick a loss bound. A snapshot loses the whole interval; an append-only log with
appendfsync everysec bounds loss to about one second.
AOF everysec + aof-use-rdb-preamble yes, RDB off via save "" — one
fork source, a ~1s bound, and the AOF rewrite still uses the compact RDB encoding for a fast restart.
EchoMQ's applied choice diverges from "enable both": AOF-only keeps a single fork source and a ~1s loss bound the checkpoints are designed against — the RDB encoding survives only as the rewrite preamble.
The durability dial — the ~1s bound is one rung
echo_store/graft.ex · the off-box rungAOF everysec is one setting on a dial, not the end of it. Durability is a choice a system turns up or
down, rung by rung: hold nothing in the volatile tier; keep an append-only log on the local volume with a bounded
loss window; replicate the record off the box so a lost machine is not a lost record. The ~1s AOF bound is the
second rung — the record survives a crash, but it lives on one machine's volume. The crash-timeline figure
contrasts that bound with the snapshot interval.
t would lose.The next rung is off-box. EchoStore.Graft is the native-BEAM replication engine: a single-writer
VolumeServer commits pages to a local CubDB store, and EchoStore.Graft.Remote.Tigris
ships segments and conditional commit objects to Tigris S3 in real time, behind a create-only commit fence so
exactly one writer claims a slot. The commit drives the bus; the archive fold drives the commit. That is the
frontier the AOF bound opens onto: the local log is rung two; commit-per-record replicated to Tigris is rung
three. EchoMQ's job HASH is ~1s-loss-tolerant, so it sits on the AOF volume; a datum that cannot lose a second
turns the dial further.
A system of record turns the durability dial to the bound its data needs — the ~1s AOF
volume for the hot job HASH, and the off-box EchoStore.Graft commit-to-Tigris rung where a lost
machine must not be a lost record.
The durable floor
Beneath the ~1s AOF bound is the floor: ETS head → Valkey bus + L2 → the local page engine
(EchoStore.Graft on CubDB) → Tigris off-box. The durability dial in full is the
Echo Persistence course — the single-writer engine, the lazy reader, and
the commit fence that ships the record off the machine.
The dial, in the architecture
The BCS persistence chapter reads the same substrate as architecture: the durability dial as a per-datum choice, coherence by mint time, and the Oban trade — jobs in the same Postgres for one transaction, versus the in-memory hot path plus the dial. The production config of this exact block is Valkey on a Fly machine.
References
sources · relatedSources
- Valkey — Persistence — RDB snapshotting, the append-only file, and the
appendfsyncloss bounds (always/everysec/no). - Valkey — BGREWRITEAOF — the background AOF rewrite that
auto-aof-rewrite-percentageschedules; the one fork source under an AOF-only posture. - Valkey — BGSAVE — the forked RDB snapshot, disabled here by
save "". - Valkey — CONFIG SET — reading and changing
appendonly,appendfsync, andsaveat runtime. - antirez — Redis design notes — the creator's notes on the snapshot-versus-log trade-off and the fork copy-on-write cost.
Related in this course
- R7.01 · Primary databasethe module hub: Redis as a system of record
- R7.01.2 · noevictionthe eviction posture that pairs with persistence
- R7.01.1 · System of recordthe row is the record; the per-datum decision
- /bcs · Persistencethe durability dial in depth
- /echo-persistencethe durable floor beneath the AOF bound