R5.03.3 · Pub/Sub vs Streams — dive
The dedicated blocking connection — a subscriber holds its lane.
The fire-and-forget side carries one operational cost the publish side hides: a subscriber
monopolizes its connection. The instant a connection issues SUBSCRIBE, it enters subscriber
mode and can no longer run ordinary commands — a GET on it is refused until it
unsubscribes. So a subscriber is not a connection borrowed from the pool; it is one given over to listening.
Grounding: echo/apps/echo_mq/lib/echo_mq/events.ex · the :conn MUST be protocol: 3 with push_to the listener (pushes arrive as {:emq_push, …}); the emq.1 resubscribe re-issues SUBSCRIBE across a reconnect.
GET while a subscriber holds its connection. A pooled subscriber drains the pool; a
dedicated lane does not.SUBSCRIBE changes the connection
valkey · subscriber modeA Redis connection in subscriber mode is a different beast. After SUBSCRIBE channel, the
connection's only job is to carry pushed messages from that channel; the protocol refuses most ordinary
commands on it until it unsubscribes. There is good reason — a subscriber must be able to receive a
message at any instant, with no request outstanding, so the connection cannot also be mid-GET.
But the consequence for connection accounting is exact: a subscription is not a quick borrow-and-return like
a GET; it holds its connection for the entire lifetime of the subscription.
This is the same hazard shape as the blocking stream read in R5.02 — a verb that holds a connection
— but it is worse in degree. A BLOCK 1000 parks for a second and re-arms; a
SUBSCRIBE parks until its owner stops listening, which may be the life of the
application. Run a subscriber on a connection drawn from the shared pool and that connection never comes
back. Run several and the pool drains: an unrelated GET elsewhere in the app waits for a
connection that is gone for good. The subscriber did not deadlock the channel; it removed connections from
circulation.
A subscriber connection refuses ordinary commands and holds for the life of the subscription. Drawn from the shared pool, it is a connection removed from circulation, not borrowed.
The fix is a dedicated lane
the dedicated-connection lawThe mitigation is the same law the blocking read follows: a verb that holds a connection gets its own
connection. Give each subscriber a connection that is not part of the shared pool, dedicated to the
subscription, and the shared pool stays whole — every ordinary GET and SET
runs on a free connection while the subscriber holds its own. It is not a tuning knob; it is a structural
requirement, because a subscriber's connection is consumed by definition, not occasionally.
In RESP3 the dedicated-lane requirement has a second half: the connection must be in protocol 3 so pushed messages arrive as out-of-band push frames the client routes to the listener, rather than tangling with ordinary replies on the wire. So a subscriber lane is specifically a RESP3 connection given over to one listening process.
A subscriber gets its own connection — structural, not optional. In RESP3 that lane is a
protocol: 3 connection so pushes arrive as out-of-band frames, not tangled in ordinary replies.
EchoMQ.Events takes it as a requirement
events.ex · init/1 · :conn protocol 3 + push_toEchoMQ.Events does not let a caller subscribe on a borrowed connection. Its :conn
must be a RESP3 connector — protocol: 3 — with push_to
set to the listener process, so pushed messages land as {:emq_push, …} mailbox frames on
a connection dedicated to the subscription. If a caller hands it connector options instead,
init/1 starts a fresh connector and sets both itself.
conn_opts = opts |> Keyword.fetch!(:connector) |> Keyword.put_new(:protocol, 3) ← RESP3: pushes arrive as push frames |> Keyword.put(:push_to, self()) ← routed to THIS listener process {:ok, c} = Connector.start_link(conn_opts)
It then subscribes once to emq:{q}:events on that lane, and a pushed message
arrives as {:emq_push, ["message", channel, payload]} — dispatched to the registered
subscriber pids and the handler. The subscription rides its own connection, so the single-owner socket the
rest of the system shares is never stalled — the same outcome the blocking stream read reaches by
holding its own lane. A listening verb on a listening connection.
Notes on ValkeyIn RESP3 a server push is a distinct reply
type, so a client can deliver a subscribed message out of band without confusing it for a command reply
— which is why the :conn is required to be protocol: 3 —
valkey.io/topics/protocol.
The resubscribe — surviving a reconnect
the emq.1 resubscribe · the loss windowA dedicated lane fixes the connection-accounting problem. A second problem remains: a connection can drop,
and a subscription does not survive a TCP reconnect on its own — the server forgets a dropped
connection's subscriptions, so a naive reconnect comes back subscribed to nothing and the feed goes silent.
The fix is the emq.1 resubscribe. The connector tracks its subscribed channels in a MapSet, and
on reconnect it re-issues the SUBSCRIBE for each, so the feed comes back live without the
listener doing anything.
That mechanism is also exactly the boundary of the at-most-once promise. Between the socket drop and the resubscribe completing, the connection is subscribed to nothing, so a message published in that window reaches no subscriber and is lost. The resubscribe narrows the window to the reconnect interval; it does not erase it, because nothing short of a durable log could. The moduledoc states both halves: the resubscribe is the mitigation, and the durable replayable receipt is the Stream, not this. A subscriber that cannot tolerate even that window does not want a subscriber — it wants the durable channel.
A subscription does not survive a reconnect on its own — the server forgets a dropped connection's
channels. Track the channels and re-issue SUBSCRIBE on reconnect to keep the feed live.
The emq.1 resubscribe holds the channels in a MapSet and re-subscribes on reconnect, so
EchoMQ.Events comes back live. The loss window between drop and resubscribe is the boundary of
at-most-once — where it matters, use the durable Stream.
The pattern, applied
echo/apps/codemojexA codemojex (echo/apps/codemojex) live-feed listener subscribes to its game's
emq:{q}:events channel on a dedicated RESP3 connector, separate from the pool the game's ordinary
reads and writes use. While it sits parked on the subscription for the life of the room, a
GET-the-board call and a SET-the-score call elsewhere in the app never wait on its
connection — the listener holds its own lane. If the connection drops for a heartbeat, the connector's
resubscribe re-issues the SUBSCRIBE and the live feed resumes on its own; the one or two events
that published during the gap are missed by the live view and lose nothing, because the durable round Stream
still holds them for the feed reader that replays the log. A dedicated lane for the live nudge, the durable
Stream for the record.
The events seam in depth
The Bus pillar teaches the broadcast channel end to end — the connector push routing, the resubscribe, and the retained log the events ride beside.
/bcs/busThe bus in the manuscript
Part B3 builds the bus the channel runs on: the emq:{q}: keyspace, jobs and lanes, and the
Stream Tier the durable record is written to.
Where the durable record endures
What a publish drops during a reconnect, the durable Stream still holds — bounded by retention and folded into the Graft floor on CubDB and on to Tigris, the durability dial taught in full.
References
sources · relatedSources
- Valkey — SUBSCRIBE — entering subscriber mode and the commands the connection then refuses.
- Valkey — Pub/Sub — the subscriber connection model and the RESP3 push delivery.
- Valkey — RESP3 protocol — the push frames a
protocol: 3connection delivers out of band. - Valkey — Cluster specification — the
{q}hash tag that places a queue's events channel on one of 16384 hash slots.
Related in this course
- R5.03 · Pub/Sub vs Streamsthe module hub
- R5.03.1 · Fire-and-forget vs durablethe at-most-once promise this connection serves
- R5.03.2 · The choosing rulewhen the fire-and-forget connection is the right choice
- R5.02.1 · The blocking readthe same dedicated-lane law for the blocking stream read
- /echomq/busthe broadcast channel and its connection seam in depth