Valkey on a Fly machine.

On Fly, the queue’s Valkey runs as its own machine, separate from the web service. The image is a pinned valkey/valkey, started with io-threads sized to the machine’s cores — one main thread executes commands, the rest feed it — a maxmemory budget, and a noeviction policy. Eviction is the wrong posture for a job store: a queue that silently drops keys is a queue that silently loses work, so memory pressure must surface as write errors and alerts. Apps reach it over the private network at echo-valkey.internal, behind a password even on the internal mesh, with auto-stop off so a machine is not reaped under a live queue.

Durability is the reason a queue chooses this engine. With appendonly on and appendfsync everysec, the volume holds an append-only log and the loss bound after a crash is roughly one second of writes, not a whole snapshot interval — and EchoMQ’s checkpoints are designed against exactly that bound. The cost is that an AOF rewrite forks, and the copy-on-write spike during the fork must be budgeted: keep maxmemory well under the machine’s RAM and schedule rewrites off-peak.

An in-memory store needs the kernel tuned away from its defaults. Transparent huge pages are turned off, because the larger pages turn a single-byte write during a fork’s copy-on-write into a large page copy and a latency spike. Memory overcommit is set so a fork does not fail when copy-on-write means little real memory is needed; swappiness is set to its lowest practical value, because a swapped page trades nanoseconds for milliseconds; and the connection backlog and file-descriptor limits are raised so a burst of clients is accepted rather than dropped.