Durable on a crash, never silently losing work.

The first question of durability is one number: after a crash, how much acknowledged work is gone. With an append-only log turned on and an fsync cadence of once a second, the answer is bounded and small.

appendonly yes makes the volume hold an append-only file — every write that changes the keyspace is appended to a log on disk. appendfsync everysec flushes that log to the disk once a second on a background thread. The manuscript states the bound directly: “with appendonly on and appendfsync everysec, the volume holds an append-only log and the loss bound after a crash is roughly one second of writes, not a whole snapshot interval.” A snapshot-only store loses everything written since the last snapshot — minutes of work; the append-only log narrows that window to about one second.

aof-use-rdb-preamble yes lets the periodic rewrite start from a compact binary preamble rather than replaying every command, so the rewrite is fast and the file stays small. save "" turns snapshotting off on purpose: with the append-only file as the single source of durability, there is no reason to keep a second snapshot fork competing for the machine. One fork source, one durability mechanism. The copy-on-write spike of that one rewrite fork is dive 1’s subject, not repeated here.

The second way a job store loses work is memory pressure. When a store fills up it must choose: throw old keys away to make room, or refuse the new write. For a cache the first is fine — an evicted key is only a future miss. For a queue it is a silent data-loss bug.

maxmemory 512mb is a loud guardrail, not a working ceiling — the live dataset is single-digit megabytes, so 512 sits far above it and leaves the rest of the box for fork copy-on-write and client buffers. maxmemory-policy noeviction is the load-bearing line. The config’s own comment names the trade: “noeviction means a runaway keyspace REJECTS writes loudly instead of being OOM-killed silently.” The manuscript states the principle: “a queue that silently drops keys is a queue that silently loses work, so memory pressure must surface as write errors and alerts.” Under noeviction, a write that would exceed maxmemory returns an error to the caller — a signal to alert on — rather than discarding a job that something already enqueued.

maxmemory-clients 64mb caps the memory a single client’s buffers may hold, so one slow or stuck consumer cannot drive the whole instance into that pressure on its own.

Rejecting writes under memory pressure is one half of failing loudly. The other half is refusing to acknowledge data the store cannot promise to keep, and giving the log a chance to flush on the way down.

propagation-error-behavior panic is the strictest possible posture, and the config states why in one line: “fail the write rather than acknowledge data that may not survive.” If a write cannot be propagated where it must go, the server fails the write rather than returning a success the data does not back. A false acknowledgement is worse than a visible error — the caller that hears “OK” stops worrying about the job; an error makes it retry.

shutdown-timeout 10 covers the orderly exit. On a SIGTERM — the signal a deploy or a platform sends to stop the process — the server is given ten seconds to flush the append-only file before the platform forces it down. The config’s comment: “give the AOF a chance to flush on SIGTERM.” Without that window, a rolling deploy could discard the last unflushed second on every restart.

A dropped connection is dropped work, so the network settings take the same posture as the persistence ones: accept the burst rather than drop it. These are the connection-survival half of the source’s kernel-tuning guide.

tcp-backlog sets the listen queue for incoming connections. The source’s kernel-tuning guide warns that the Linux default is 128, so during a traffic spike new connections are dropped before the server ever sees them. The effective backlog is min(tcp-backlog, somaxconn) — both the application’s tcp-backlog and the kernel’s net.core.somaxconn must be raised, or whichever is smaller caps the queue. tcp-backlog lives in valkey.conf; somaxconn and the other knobs are kernel settings, set on the host:

The source sets somaxconn to 65536 so the kernel’s queue is never the limit, and raises the file-descriptor ceiling to 65536 because each connection consumes one file descriptor and a server’s maxclients is bounded by the descriptors available to it. Back in valkey.conf, tcp-keepalive 300 sends a keepalive every 300 seconds so a load balancer or firewall does not silently reap a long-lived idle connection, and the pub/sub output buffer is bounded so a slow subscriber is dropped rather than allowed to grow the server’s memory without limit.

The settings above bound the queue’s crash loss to about one second. That is the right floor for a volatile work tier, but it is not the whole durability story. Deep history and off-box durability live one tier down, on the persistence floor: when the bus trims a stream, EchoStore.StreamArchive folds the trimmed segments into EchoStore.Graft’s append-only CubDB at a reserved high page range, and a streamer ships those pages on to Tigris object storage — deep history without resident memory. The volatile tier keeps the last second; the durable floor keeps everything, addressable beside the live tail.

A durable job store is one that fails loudly. The append-only log with appendfsync everysec bounds crash loss to about a second; noeviction makes memory pressure a rejected write and an alert instead of a silently dropped key; propagation-error-behavior panic refuses to acknowledge data that may not survive; shutdown-timeout 10 flushes the log on the way down; and a raised tcp-backlog/somaxconn pair and file-descriptor ceiling accept a connection burst rather than drop it. Every one of these settings is in the real valkey.conf (or the host sysctls beside it) that fronts the codemojex queue. This is the last dive of the module — the recap returns to the chapter.