B8.3 · dive 3 of 4 · Production on Fly
EchoMQ setup and monitoring.
EchoMQ is wired into the supervision tree as one bus and connector that negotiates RESP3, with lane consumers — score, settle, notify, commands — on a heartbeat and a lease; you watch it by queue depth, the dashboard task, and lease health.
The idea
codemojex · echomq, set up and watchedEchoMQ is wired into the supervision tree, not bolted on. The application starts a shared bus and its connector against Valkey on the configured port, and each connector negotiates the RESP3 protocol with HELLO 3 and pipelines its commands rather than waiting on each round trip. The bus is the one place that holds the Valkey connection; everything that touches the queue goes through it, so there is a single, supervised path to the store.
Above the bus sit the consumers, each a supervised child draining one lane: the scoring authority, the settlement worker, the notification worker, and the inbound-command worker. Each runs on a heartbeat and takes a lease on the job it is working, so a crashed consumer’s in-flight job becomes visible again rather than lost, and the supervisor restarts the consumer in place. The lanes keep a hot player from starving the rest, the same fair rotation the play path relies on.
What you watch in production is the depth of the lanes and the health of the consumers. Queue length is a direct read, the bundled dashboard task connects to the same Valkey and prints the queues and their backlog, and the lease and heartbeat timings tell you whether a consumer is keeping up. Because the store is set to refuse rather than evict, a backlog that grows without draining surfaces as pressure to alert on — a signal that a consumer is stuck or the work is outrunning it — rather than as work quietly disappearing.
The doors
the sibling coursesEchoMQ — the protocol, in depth
The bus B3 narrates, taught rung by rung: the keyspace, the Lua inventory, and the 3.0 Stream Tier — XADD, XREADGROUP, XACK — with conformance on Valkey.
Redis Patterns Applied
The substrate patterns under the bus: sorted sets, atomic Lua moves, locks, and streams — the judgement layer.
/echo-persistenceEcho Persistence — the durable floor
Beneath the volatile tiers: the Graft page engine built twice, the commit-LSN loop, and the stream archive on Tigris. Where Part I says state lives somewhere, this says where it lives durably.
References
sources · relatedSources
- Kreps — The Log: real-time data’s unifying abstractionthe lanes a single consumer drains, watched by their depth
- The Go Blog — Share Memory By Communicatingthe consumers draining work by message, and the deps an app keeps to itself
- Erlang/OTP — the gen_server behaviourthe supervised processes the release boots as one application
Related
- BCS — the course homethe law, the id anatomy, the chapter map
- Echo Persistencethe durable floor beneath the bus