jonnify·redis /redis-patterns/streams-events/pubsub/fire-and-forget-vs-durable

R5.03.1 · Pub/Sub vs Streams — dive

Fire-and-forget vs durable — what the log remembers.

A PUBLISH and an XADD look alike from the caller, but they make opposite promises about a consumer who is not there yet. A publish reaches the subscribers present at that instant and nobody else; the log remembers an append for the reader who arrives a minute later. This dive sits the two side by side — and reads the gap between them.

Grounding: echo/apps/echo_mq/lib/echo_mq/events.ex · publish/5 (PUBLISH emq:{q}:events, the discarded subscriber count) and stream.ex · append/4 (the {:ok, branded} receipt) — at-most-once against durable.

One event, two channels, a late reader. Select a channel to follow what a reader who connects after the event sees. The green dot is when the event happens; the reader joins to its right.
event reader joins PUBLISH sent now sees none XADD appended in log reads it
readout · Select a channel. A reader who joins after the event sees nothing on the publish — it was not stored — but reads the entry on the Stream, because the log kept it.
§1

What a PUBLISH guarantees

valkey · publish

PUBLISH channel payload is a delivery, not a store. Valkey sends the payload to every connection subscribed to channel at the moment of the call and returns an integer — the number of subscribers it reached. That integer is the whole receipt, and it is weak: a publish that reaches 0 returns 0, which the sender cannot tell apart from a healthy delivery without out-of-band knowledge of who should be listening. There is no buffer behind the channel and no cursor in front of it. A subscriber that connects one millisecond after the publish sees nothing of it; a subscriber that drops for a heartbeat and reconnects has missed whatever published while it was gone.

This is at-most-once: a present subscriber receives a message zero or one times, and the message is never held for an absent one. It is the right promise for a signal whose value is now — a live progress tick, a presence ping, a cache-invalidation nudge — where a missed message means only that the next one supersedes it. It is the wrong promise for anything that must be remembered. The trap is that the API hides the difference: PUBLISH looks like it sent something durable, and the returned count looks like a confirmation.

A publish reaches present subscribers and stores nothing. Its return value — a subscriber count — cannot tell a delivery to nobody from a delivery to one. At-most-once, by construction.

§2

What an XADD keeps

valkey · xadd

XADD key * field value … is a store with a delivery on top. Valkey appends the entry, assigns it a monotonic id, and keeps it in the log until a retention policy removes it. The return value is that id — a durable handle the caller keeps, not a count it has to interpret. A reader reads with XRANGE from any position: the start of the log (-), a saved cursor, or only-new (> under a consumer group). So a reader that was absent reads what it missed, a reader that crashed resumes from its cursor, and a reader rebuilding state replays from -. The log is the source of truth, and the id is how every reader agrees on order.

The cost of that durability is memory and a retention decision — a log that only grows is a leak, so a Stream needs a trim policy where a channel needs nothing. That cost is the subject of R5.02's trimming dive. Here the point is narrower: where a publish answers a subscriber count and forgets, an append answers a receipt and remembers.

What each call hands back. Select a call to read its return value and what the caller can do with it. A count cannot be replayed; a receipt can be read back.
PUBLISH ch payload XADD key * … subscriber count0 == reached nobody branded entry iddurable · read back later
readout · Select a call. A publish returns a subscriber count — weak, and 0 when nobody heard. An append returns the entry's id — a durable receipt a reader can read back from the log.

A publish hands back a count it cannot interpret. An append hands back an id the log keeps — a receipt, not a confirmation, and the difference is durability.

§3

EchoMQ — the count discarded, the receipt kept

events.ex · publish/5 · stream.ex · append/4

EchoMQ.Events does not pretend the channel is durable. publish(conn, queue, event, job_id, extra \\ []) gates the id at the key builder, encodes the cjson payload, and issues PUBLISH emq:{q}:events — best-effort, a no-op when no one is listening. The returned subscriber count is discarded on purpose: retrying a publish that reached zero subscribers would not change the promise, because there is still no store behind the channel.

echo/apps/echo_mq/lib/echo_mq/events.ex · publish/5, the subscriber count discardedverbatim
case Connector.command(conn, ["PUBLISH", channel(queue), payload]) do
  {:ok, _n} -> :ok            ← the subscriber count, discarded — best-effort
  other -> other
end

EchoMQ.Stream.append/4 is the durable contrast on the same bus. It mints an EVT-branded record id host-side, appends the XADD, and returns {:ok, branded} — the moduledoc puts it plainly: the branded id is the receipt. The entry now exists in the log and stays there until retention removes it.

echo/apps/echo_mq/lib/echo_mq/stream.ex · append_id/5, the branded id returned as the receiptverbatim
case Connector.command(conn, parts) do
  {:ok, id} when is_binary(id) -> {:ok, branded}   ← the receipt: the branded id
  {:ok, {:error_reply, @id_too_small}} -> {:error, :nonmonotonic}
  {:ok, {:error_reply, msg}} -> {:error, {:error_reply, msg}}
  {:error, _} = err -> err
end

Notes on ValkeyPUBLISH answers an integer reply — the count of clients that received the message — while XADD answers the new entry's id string, a value a later XRANGE can read back — valkey.io/commands/xadd.

§4

The gap — the no-live-subscriber case

the difference, in one event

Put one event through both channels with no subscriber present, and the difference is the whole module in miniature. The publish reaches a subscriber count of zero and is gone — there is nowhere it could have been stored. The append writes an entry that sits in the log, and the reader who connects a minute later reads it in full. Neither is wrong; they are different promises. The publish is a doorbell that only the people home hear; the append is a letter in the box that waits for whoever checks the mail.

the pattern

A PUBLISH with no live subscriber is lost — at-most-once, nothing stored. An XADD with no reader is read the moment a reader arrives — the log kept it.

its EchoMQ application

Append the event to the durable Stream so the record exists and can be replayed; publish a lighter notification on Events so a live view reacts at once. The Stream is the system of record; the publish is the live nudge.

One event, two channels: the publish forgets the instant no one is listening; the append is held for the reader who arrives later. Use both — durable for the record, live for the reaction.

§5

The pattern, applied

echo/apps/codemojex

In codemojex (echo/apps/codemojex) a round-settled event takes exactly that dual path. The settlement is appended to the round's durable Stream with EchoMQ.Stream.append/4, so the {:ok, branded} receipt records that it happened and a feed reader can replay the round or resume after a restart. A lighter "round settled" notification is published on EchoMQ.Events so a live Telegram view updates the instant the round closes. A player who is not watching at that instant misses the live nudge — and loses nothing, because the durable Stream still holds the settlement for the feed that reads it later. Durable where the record matters, live where the reaction does.

/echomq/bus

Both channels, in depth

The Bus pillar teaches the broadcast channel and the retained, replayable log end to end: the events seam, the consumer-group loop, and the receipt the writer returns.

 /bcs/bus

The Stream Tier in the manuscript

Part B3 builds the bus the durable channel runs on: the emq:{q}: keyspace, jobs and lanes, and the Stream Tier the receipt is written to.

 /echo-persistence

Where the durable log endures

What the Stream retains is bounded by retention; trimmed segments fold into the Graft floor on CubDB and on to Tigris — the durability dial taught in full.
§6

References

sources · related

Sources

Related in this course