jonnify·bcs /bcs/fly/echomq-setup-and-monitoring

B8.3 · dive 3 of 4 · Production on Fly

EchoMQ setup and monitoring.

EchoMQ is wired into the supervision tree as one bus and connector that negotiates RESP3, with lane consumers — score, settle, notify, commands — on a heartbeat and a lease; you watch it by queue depth, the dashboard task, and lease health.

A supervised bus and connector, four lane consumers, and the signals you monitor. Select a part.
the bus + connectordials the Valkey portHELLO 3, pipelinedone supervised path the lane consumersscore · settlenotify · commandsa heartbeat and a lease monitoringqueue depththe dashboard taskalert on backlog
readout · Select a part. One supervised bus and connector dial Valkey and negotiate RESP3; four lane consumers run on a heartbeat and a lease; monitoring watches queue depth, the dashboard task, and lease timings, alerting on a growing backlog.
§1

The idea

codemojex · echomq, set up and watched

EchoMQ is wired into the supervision tree, not bolted on. The application starts a shared bus and its connector against Valkey on the configured port, and each connector negotiates the RESP3 protocol with HELLO 3 and pipelines its commands rather than waiting on each round trip. The bus is the one place that holds the Valkey connection; everything that touches the queue goes through it, so there is a single, supervised path to the store.

Above the bus sit the consumers, each a supervised child draining one lane: the scoring authority, the settlement worker, the notification worker, and the inbound-command worker. Each runs on a heartbeat and takes a lease on the job it is working, so a crashed consumer’s in-flight job becomes visible again rather than lost, and the supervisor restarts the consumer in place. The lanes keep a hot player from starving the rest, the same fair rotation the play path relies on.

What you watch in production is the depth of the lanes and the health of the consumers. Queue length is a direct read, the bundled dashboard task connects to the same Valkey and prints the queues and their backlog, and the lease and heartbeat timings tell you whether a consumer is keeping up. Because the store is set to refuse rather than evict, a backlog that grows without draining surfaces as pressure to alert on — a signal that a consumer is stuck or the work is outrunning it — rather than as work quietly disappearing.

§4

The doors

the sibling courses
§2

References

sources · related

Sources

Related