The events channel · dive 02
Publish after the verdict
The announcement belongs to the host, not the script that decides.
An event is published after a transition decides, from the host — not inside
the script that decides. publish/5 gates the id at the key builder, builds a small cjson
object by hand, and issues one PUBLISH emq:{q}:events direct through
EchoMQ.Connector.command/3 — no new script, no Lua. That one placement choice is
what keeps the byte-frozen transition scripts byte-unchanged. See the choice on the right, then
read the code.
Why host-side · two placements
Where the publish lives — and why it is host-side
A queue transition (complete, retry, the stalled reaper) is an
atomic Lua script. The publish does not go inside it. The calling code runs the transition, reads
its verdict, and then — host-side, in Elixir — calls publish/5 to announce what
happened. The reason is a hard invariant of this bus: the transition scripts are byte-frozen.
Adding a PUBLISH to a script would change its bytes (and its SHA), which is exactly what
the freeze forbids. Publishing from the host keeps every script byte-for-byte identical to what shipped, while
still broadcasting the outcome.
(One event is published inline because its transition already does: progress. The
@update_progress seam established the emq:{q}:events channel and
the cjson payload shape; every other lifecycle event reuses that channel from the host.)
# echo_mq — EchoMQ.Events.publish/5 (host-side, after the verdict) # Announce a lifecycle event AFTER its transition has decided. The transition # scripts are byte-frozen, so the publish lives here in the host, not in the # script — the scripts stay byte-unchanged. One PUBLISH, issued DIRECT (no Lua). def publish(conn, queue, event, job_id, extra \ []) do # gate the id at the key builder (INV5) — raises on an ill-formed id BEFORE the # wire, so a malformed JOB id can never reach a PUBLISH _ = Keyspace.job_key(queue, job_id) payload = encode_event(event, job_id, extra) case Connector.command(conn, ["PUBLISH", channel(queue), payload]) do {:ok, _n} -> :ok # best-effort — n is the receiver count (may be 0) other -> other end end # the channel — emq:{q}:events, the SAME suffix the @update_progress seam publishes on def channel(queue), do: Keyspace.queue_key(queue, "events")
Two facts carry weight. First, the id is gated at the key builder before anything is sent:
Keyspace.job_key(queue, job_id) raises if the job id is ill-formed, so a malformed branded
id cannot reach a PUBLISH. The return value is discarded — the call exists purely
for its gate. Second, the publish is best-effort: PUBLISH answers the
number of receivers, which the function throws away and maps to :ok. Zero receivers is
success, not error.
Split the work: the script decides, the host announces. That one line of separation is what lets the bus broadcast every outcome without ever editing a frozen transition.
The payload — flat cjson, built by hand
The payload is a tiny JSON object of flat string fields: the event name, the job id, and any
extra keyword pairs (e.g. [progress: "50"]). The bus carries
no JSON dependency, so the object is built by string concatenation, with the two
JSON-significant characters escaped.
# echo_mq — EchoMQ.Events.encode_event/3 (the flat cjson payload, built by hand) # {"event":"<name>","job":"<id>", …extra} — flat string fields only. No JSON # library on the bus; the object is concatenated, with \ and " escaped. defp encode_event(event, job_id, extra) do fields = [{"event", to_string(event)}, {"job", job_id}] ++ Enum.map(extra, fn {k, v} -> {to_string(k), to_string(v)} end) body = fields |> Enum.map(fn {k, v} -> ~s("#{k}":"#{escape(v)}") end) |> Enum.join(",") "{" <> body <> "}" end defp escape(v) do v |> String.replace("\\", "\\\\") # backslash first |> String.replace("\"", "\\\"") # then the quote end
No JSON library on the bus — only a concatenated object with the two significant characters escaped. Small payloads, zero dependency.
Reading the name back — a scan, not a parser
On the receive side, the consumer needs the event name. It does not parse the JSON. Because the
payload is hand-built and cjson key order is not guaranteed, the name is read by a substring scan
for the event field, and the matched string is resolved with
String.to_existing_atom/1. That last choice is the safety one: an unknown name answers
:unknown rather than minting an atom from the wire. Atoms are not garbage
collected; turning arbitrary wire bytes into atoms is an unbounded-memory hazard, so the function only ever returns
an atom that already exists.
# echo_mq — EchoMQ.Events.event_name/1 (read the name by scan; never mint an atom) # The bus has no JSON parser and cjson key order is not guaranteed — so read the # "event" field by substring scan. Resolve with to_existing_atom: an UNKNOWN # name answers :unknown, never minting an atom from the wire (atoms aren't GC'd). def event_name(payload) when is_binary(payload) do case Regex.run(~r/"event"\s*:\s*"([^"]+)"/, payload) do [_, name] -> try do String.to_existing_atom(name) # only an atom that already exists rescue ArgumentError -> :unknown # unknown name → :unknown, not a new atom end _ -> :unknown end end
Read the name by scan, resolve it to an existing atom or :unknown — never
mint an atom from the wire. A hand-built payload deserves a hand-built, safe read.
The publish round-trip
Pick a verdict and watch the whole round-trip: publish/5 gates the id, builds the cjson
payload with encode_event/3, and issues PUBLISH emq:{q}:events;
then event_name/1 scans the same payload back to the atom. The
unknown option injects a name no atom exists for, so the read answers
:unknown. The strings shown are exactly what the code produces — pure, no network.
publish/5 → encode_event/3 → event_name/1
The id is gated before the payload is built — an ill-formed job id raises at
job_key/2 and never reaches the wire. The read is a scan, so it tolerates any key order and
refuses any name it does not already know.
Pattern & implementation
Emit a domain event when state changes, so other parts of the system can react without coupling to the writer. R5 · Streams & Events teaches the events side of the pattern.
publish/5 emits after the transition's verdict, host-side, gating the
id at the key builder and issuing one PUBLISH emq:{q}:events direct — so the
byte-frozen transition scripts stay byte-unchanged.
The pattern says emit on the change. The implementation emits from the host after the change, so the change's script never has to move.
Recap — the host announces
A lifecycle event is published host-side, after a transition's verdict, by
publish/5 — one PUBLISH emq:{q}:events through
EchoMQ.Connector.command/3, no Lua. The id is gated at the key builder before the wire;
the cjson payload is built by hand (no JSON dependency); the receiver reads the name by substring scan and
resolves it with String.to_existing_atom/1, answering :unknown
rather than minting an atom. The next dive: the guarantee the channel makes — at-most-once — and its
control-plane sibling, EchoMQ.Cancel.
Next in the module: Fire-and-forget — what at-most-once costs, the resubscribe and the stream as the two mitigations, and the cooperative cancel token.
References
Where the host-side publish, the hand-built payload, and the safe name read are grounded, and the command beneath them.
Sources
- Valkey — PUBLISH — the single command
publish/5issues; its reply is the receiver count, discarded for best-effort. - Valkey — Introduction to Streams — the bus surfaces this pillar builds on.
- Valkey — Cluster specification — the
{q}hashtag keeps the events channel on the queue's slot.
Related in this course
- The events channel — the module this dive belongs to.
- Subscribe and handle — the receive side that reads these payloads.
- Fire-and-forget — the at-most-once guarantee of this publish.
- redis-patterns · Streams & Events — the events pattern.