Codemojex · dive C8.2
The pragmatic Valkey node.
Valkey runs commands on one thread, and that single fact sizes the machine: the node is provisioned for a steady latency tail, not for throughput headroom. The whole competitive state and the EchoMQ bus live on one small, sharply-configured machine — the dedicated echo-valkey Fly node — private on the org’s 6PN, upgraded only on evidence.
Grounding — design §The pragmatic Valkey node · echo/config/runtime.exs (the echo-valkey wiring) · application.ex (conn_opts/0 · valkey_host/0).
everysec AOF fsync on its own thread, jemalloc’s background purge, the lazy frees.Four choices, one budget
design §The pragmatic Valkey nodeThe recommendation is short because the constraint is sharp. The launch node is Valkey 9.1 over its bundled jemalloc on a Fly shared-cpu-2x, one-gigabyte machine, eviction off. The command thread is the budget every choice protects: a guess costs a handful of Lua steps on that thread, and the tail stays steady as long as nothing else competes for it — not an idle I/O thread burning the burst balance, not a fork the machine cannot absorb, not an eviction scan.
maxmemory is a loud 512-megabyte guardrail under noeviction — far above a working set that is single-digit megabytes, because player balances live in Postgres, not here. The point of the guardrail is the failure mode it buys: a runaway keyspace rejects writes rather than being killed. Durability is AOF alone, one fork source; and jemalloc is the allocator the resident-size budget depends on. The node is private by construction — bound to the IPv6 wildcard so it answers only on the org’s 6PN, with requirepass on top.
Every figure in this section is the design’s: §The pragmatic Valkey node records the machine class, the 512-megabyte guardrail, the io-threads 1 rationale, and the AOF-alone durability posture. One caveat travels with the node: its volume lives on one host in one region — the single point of failure to retire with a replica before real money flows.
The wiring — env in, IPv6 out
runtime.exs · application.exThe connection is wired from the environment at release boot. runtime.exs reads three variables — the host, the password, and a port that defaults to 6390. Unset locally, the connectors fall back to 127.0.0.1:6390 with no auth, so dev and test run against a local engine unchanged.
# The shared Valkey — the EchoMQ bus + the real-time competitive state — lives on the dedicated
# echo-valkey Fly node (private 6PN, requirepass). Wire host/port/password from the env. If
# VALKEY_HOST is unset (local dev/test), the connectors fall back to 127.0.0.1:6390 with no auth.
if valkey_host = System.get_env("VALKEY_HOST") do
config :codemojex, valkey_host: String.to_charlist(valkey_host)
end
if valkey_password = System.get_env("VALKEY_PASSWORD") do
config :codemojex, valkey_password: valkey_password
end
config :codemojex, valkey_port: String.to_integer(System.get_env("VALKEY_PORT") || "6390")
The app side dials over Fly’s IPv6-only 6PN. Codemojex.Application builds one shared conn_opts list — RESP3 plus the port, then the optional host and password — for every Valkey lane: the Bus, the near-cache tables, and each consumer’s private lane. The host goes through valkey_host/0, which resolves the configured hostname to its AAAA record before the connector ever dials.
# The shared connector options for every Valkey lane (the Bus, the near-cache tables, and each # consumer's private lane). host/password come from prod config (runtime.exs reads the env); # unset locally → omitted, so the connector uses its 127.0.0.1:6390 no-auth default (dev/test). defp conn_opts do [protocol: 3, port: Application.get_env(:codemojex, :valkey_port, 6390)] |> maybe_put(:host, valkey_host()) |> maybe_put(:password, Application.get_env(:codemojex, :valkey_password)) end # Resolve the configured Valkey host to an IPv6 address tuple so the connector dials echo-valkey # over Fly's IPv6-only 6PN: gen_tcp infers inet6 from an 8-element address tuple, so this needs # no global :kernel/inet config (which a release cannot set at compile time) and no connector # change. Unset (local dev/test) → nil → the connector falls back to 127.0.0.1. A charlist # hostname is resolved to its AAAA; if resolution fails the charlist passes through (best effort). defp valkey_host do case Application.get_env(:codemojex, :valkey_host) do nil -> nil host -> case :inet.getaddr(host, :inet6) do {:ok, addr} -> addr {:error, _} -> host end end end
The trick is in the return shape: gen_tcp infers inet6 from an 8-element address tuple, so the release needs no global :kernel/inet configuration — which a release cannot set at compile time — and no connector change. On a resolve failure the charlist passes through, best effort.
The upgrade ladder
each step on evidenceThe launch node is deliberately small, and the path out of it is a ladder, not a rebuild. Each step is taken on evidence, never ahead of it — and the order matters, because the first step retires the only risk that touches money.
The last step is the one the keyspace prepared for on day one: the bus’s keys are braced — emq:{q}: — so every key of a queue hashes to one slot, and a cluster split moves whole queues, never fragments of one. Grounding — design §The pragmatic Valkey node (the ladder) · the v2 keyspace law taught in C3.
Two version-9 touchpoints
forward · not shippedExpiry at close — forward. The closed-game keys — the board, the hashes, the counter, the marker, the lock hashes — are not expired by the close today, so under noeviction they accumulate. A key-level expiry at close would fix it on any version, and HEXPIRE would give per-field control where a hash should outlive some of its fields. The lock plane’s marker could collapse to one self-expiring hash field, released by token with DELIFEQ rather than an unconditional delete. None of this is shipped surface; the design records it as the maintenance the guardrail eventually collects.
The 9.1 fixes. Two land on paths the bus uses directly: the stream-trim null-deref on EchoMQ.Stream.trim, and the rehashing latency reduction on the lane sorted sets and the job-hash dictionary as they grow. The version choice is the same discipline as the sizing: taken for a named path the system exercises, not for a headline.
Grounding — design §The pragmatic Valkey node (the two touchpoints) · EchoMQ.Stream.trim/4 in echo/apps/echo_mq/lib/echo_mq/stream.ex.
References
sources · relatedSources
- Valkey — Benchmarkingsingle-threaded execution — the sizing rationale
- Valkey — Persistencethe append-only file and the
everysecloss bound - Valkey — 9.1.0 release notesthe fixes on the bus’s paths
- Fly.io — Machine sizingthe shared-cpu class and the burst balance
Related
- C8 — Production Deploymentthe chapter this dive belongs to
- Redis Patterns Appliedthe Valkey patterns this node serves
- EchoMQ — the coursethe bus riding this node