jonnify·redis /redis-patterns/streams-events/pubsub/the-dedicated-blocking-connection

R5.03.3 · Pub/Sub vs Streams — dive

The dedicated blocking connection — a subscriber holds its lane.

The fire-and-forget side carries one operational cost the publish side hides: a subscriber monopolizes its connection. The instant a connection issues SUBSCRIBE, it enters subscriber mode and can no longer run ordinary commands — a GET on it is refused until it unsubscribes. So a subscriber is not a connection borrowed from the pool; it is one given over to listening.

Grounding: echo/apps/echo_mq/lib/echo_mq/events.ex · the :conn MUST be protocol: 3 with push_to the listener (pushes arrive as {:emq_push, …}); the emq.1 resubscribe re-issues SUBSCRIBE across a reconnect.

Where the subscriber parks. Select a topology to read what happens to an unrelated GET while a subscriber holds its connection. A pooled subscriber drains the pool; a dedicated lane does not.
shared pool SUBSCRIBE · held for the life of the feed free free GET on a pool one connection short dedicated lane + whole pool own lane · SUBSCRIBE (protocol 3) free free GET runs at once
readout · Select a topology. A subscriber holds its connection for the whole life of the feed — from the shared pool that removes a connection from circulation; on its own lane the shared pool stays whole.
§1

SUBSCRIBE changes the connection

valkey · subscriber mode

A Redis connection in subscriber mode is a different beast. After SUBSCRIBE channel, the connection's only job is to carry pushed messages from that channel; the protocol refuses most ordinary commands on it until it unsubscribes. There is good reason — a subscriber must be able to receive a message at any instant, with no request outstanding, so the connection cannot also be mid-GET. But the consequence for connection accounting is exact: a subscription is not a quick borrow-and-return like a GET; it holds its connection for the entire lifetime of the subscription.

This is the same hazard shape as the blocking stream read in R5.02 — a verb that holds a connection — but it is worse in degree. A BLOCK 1000 parks for a second and re-arms; a SUBSCRIBE parks until its owner stops listening, which may be the life of the application. Run a subscriber on a connection drawn from the shared pool and that connection never comes back. Run several and the pool drains: an unrelated GET elsewhere in the app waits for a connection that is gone for good. The subscriber did not deadlock the channel; it removed connections from circulation.

A subscriber connection refuses ordinary commands and holds for the life of the subscription. Drawn from the shared pool, it is a connection removed from circulation, not borrowed.

§2

The fix is a dedicated lane

the dedicated-connection law

The mitigation is the same law the blocking read follows: a verb that holds a connection gets its own connection. Give each subscriber a connection that is not part of the shared pool, dedicated to the subscription, and the shared pool stays whole — every ordinary GET and SET runs on a free connection while the subscriber holds its own. It is not a tuning knob; it is a structural requirement, because a subscriber's connection is consumed by definition, not occasionally.

In RESP3 the dedicated-lane requirement has a second half: the connection must be in protocol 3 so pushed messages arrive as out-of-band push frames the client routes to the listener, rather than tangling with ordinary replies on the wire. So a subscriber lane is specifically a RESP3 connection given over to one listening process.

A subscriber gets its own connection — structural, not optional. In RESP3 that lane is a protocol: 3 connection so pushes arrive as out-of-band frames, not tangled in ordinary replies.

§3

EchoMQ.Events takes it as a requirement

events.ex · init/1 · :conn protocol 3 + push_to

EchoMQ.Events does not let a caller subscribe on a borrowed connection. Its :conn must be a RESP3 connector — protocol: 3 — with push_to set to the listener process, so pushed messages land as {:emq_push, …} mailbox frames on a connection dedicated to the subscription. If a caller hands it connector options instead, init/1 starts a fresh connector and sets both itself.

echo/apps/echo_mq/lib/echo_mq/events.ex · init/1, the dedicated RESP3 lane built when none is givenverbatim
conn_opts =
  opts
  |> Keyword.fetch!(:connector)
  |> Keyword.put_new(:protocol, 3)        ← RESP3: pushes arrive as push frames
  |> Keyword.put(:push_to, self())        ← routed to THIS listener process

{:ok, c} = Connector.start_link(conn_opts)

It then subscribes once to emq:{q}:events on that lane, and a pushed message arrives as {:emq_push, ["message", channel, payload]} — dispatched to the registered subscriber pids and the handler. The subscription rides its own connection, so the single-owner socket the rest of the system shares is never stalled — the same outcome the blocking stream read reaches by holding its own lane. A listening verb on a listening connection.

Notes on ValkeyIn RESP3 a server push is a distinct reply type, so a client can deliver a subscribed message out of band without confusing it for a command reply — which is why the :conn is required to be protocol: 3 — valkey.io/topics/protocol.

§4

The resubscribe — surviving a reconnect

the emq.1 resubscribe · the loss window

A dedicated lane fixes the connection-accounting problem. A second problem remains: a connection can drop, and a subscription does not survive a TCP reconnect on its own — the server forgets a dropped connection's subscriptions, so a naive reconnect comes back subscribed to nothing and the feed goes silent. The fix is the emq.1 resubscribe. The connector tracks its subscribed channels in a MapSet, and on reconnect it re-issues the SUBSCRIBE for each, so the feed comes back live without the listener doing anything.

The loss window across a reconnect. Select a segment to read what the feed is doing. Between the drop and the resubscribe the connection is subscribed to nothing — a publish in that window is lost.
subscribed · feed live gap: no sub resubscribed · live SUBSCRIBE once socket drop SUBSCRIBE re-issued
readout · Select a segment. While subscribed the feed is live; between the drop and the resubscribe the connection is subscribed to nothing, so a publish in that window is lost; then the resubscribe restores the feed.

That mechanism is also exactly the boundary of the at-most-once promise. Between the socket drop and the resubscribe completing, the connection is subscribed to nothing, so a message published in that window reaches no subscriber and is lost. The resubscribe narrows the window to the reconnect interval; it does not erase it, because nothing short of a durable log could. The moduledoc states both halves: the resubscribe is the mitigation, and the durable replayable receipt is the Stream, not this. A subscriber that cannot tolerate even that window does not want a subscriber — it wants the durable channel.

the pattern

A subscription does not survive a reconnect on its own — the server forgets a dropped connection's channels. Track the channels and re-issue SUBSCRIBE on reconnect to keep the feed live.

its EchoMQ application

The emq.1 resubscribe holds the channels in a MapSet and re-subscribes on reconnect, so EchoMQ.Events comes back live. The loss window between drop and resubscribe is the boundary of at-most-once — where it matters, use the durable Stream.

§5

The pattern, applied

echo/apps/codemojex

A codemojex (echo/apps/codemojex) live-feed listener subscribes to its game's emq:{q}:events channel on a dedicated RESP3 connector, separate from the pool the game's ordinary reads and writes use. While it sits parked on the subscription for the life of the room, a GET-the-board call and a SET-the-score call elsewhere in the app never wait on its connection — the listener holds its own lane. If the connection drops for a heartbeat, the connector's resubscribe re-issues the SUBSCRIBE and the live feed resumes on its own; the one or two events that published during the gap are missed by the live view and lose nothing, because the durable round Stream still holds them for the feed reader that replays the log. A dedicated lane for the live nudge, the durable Stream for the record.

§6

References

sources · related

Sources

Related in this course