jonnify·redis /redis-patterns/queues/blocking-vs-polling/the-marker-wake-up

R3.05.3 · Blocking vs polling — dive 3

The wake-up doorbell.

EchoMQ's consumer parks on a dedicated wake key. The admit script rings it with one LPUSH, the parked block returns, and the consumer drains — one ring returns one worker.

Grounding: EchoMQ.Consumer.park/1 runs BLPOP emq:{queue}:wake; the admit/promote/reclaim Lua runs LPUSH emq:{queue}:wake '1' + LTRIM emq:{queue}:wake 0 63 (verified in jobs.ex, lanes.ex, stalled.ex). The pool's one-blocker EchoMQ.Metronome, all real in echo/apps/echo_mq. The worked consumer is codemojex.

The doorbell handshake. Select a step to light it: the consumer parks, the admit rings the wake key, the parked block returns.
produceradmit job wake keyemq:{q}:wake consumerparked LPUSH '1' BLPOP returns Lanes.claim
readout · Select a step. The admit rings the wake key with LPUSH; the parked BLPOP returns and the consumer drains.

Static verdicts — ring → LPUSH emq:{queue}:wake '1', capped by LTRIM … 0 63 · return → one BLPOP returns, then Lanes.claim.

§1

The park

consumer.ex · park/1

EchoMQ parks on a dedicated wake key, not on the work list. The consumer's loop is reap → promote → drain → park, repeating: it reaps expired leases, promotes due schedules, drains the ring, then parks on the wake key until readiness arrives as a wake or the beat elapses. Decoupling the signal from the work means one wake fires no matter which lane the new work landed on.

echo_mq/consumer.ex · park/1 — park on the wake key for one beatfrozen
defp park(s) do
  secs = :erlang.float_to_binary(s.beat_ms / 1000, decimals: 3)
  wake = Keyspace.queue_key(s.queue, "wake")          -- emq:{queue}:wake
  _ = Connector.command(s.conn, ["BLPOP", wake, secs], s.beat_ms + 2_000)
  :ok
end

The key is built by EchoMQ.Keyspace.queue_key(queue, "wake") → emq:{queue}:wake, hash-tagged on the queue name so it shares the queue's cluster slot. BLPOP returns when a wake token is pushed, or once when the beat elapses.

The consumer parks on BLPOP emq:{queue}:wake <beat> — one outstanding round trip, zero traffic until a wake or the beat.

§2

Ringing the doorbell

jobs.ex / lanes.ex / stalled.ex · LPUSH wake

The producer rings the wake key from inside the admit Lua. Pushing a job onto the ring runs two calls — a push and a trim:

# inside the admit / promote / reclaim script (jobs.ex, lanes.ex, stalled.ex)
redis.call('LPUSH', base .. 'wake', '1')       # ring: return any parked BLPOP
redis.call('LTRIM', base .. 'wake', 0, 63)     # bound the list to 64 tokens

The LPUSH returns the parked BLPOP; the LTRIM keeps the wake key from growing without bound when wakes arrive faster than a consumer drains them. The same two calls appear wherever new work becomes serviceable — an enqueue, a schedule promotion, a reclaimed stalled lease — so a parked consumer is returned by every path that adds work, not only a fresh enqueue.

The wake key carries no payload — the token is a constant '1'. BLPOP removes one element, so one LPUSH returns exactly one parked block. The actual job is never on the wake key; it is in the ring and the lanes, and the returned consumer fetches it with EchoMQ.Lanes.claim/3. The wake key only answers "is there something to drain," and the drain answers "what." That separation is why the bounded LTRIM is safe: a few extra tokens cost nothing, because no token is a job.

LPUSH wake '1'
The ring. One constant token; returns one parked BLPOP.
LTRIM wake 0 63
The bound. Caps the wake list at 64 tokens so it never grows unbounded.
the token
A signal, not a job — the job stays in the ring; the consumer claims it on return.

One LPUSH emq:{queue}:wake '1' returns one parked BLPOP; the LTRIM bounds the signal, and the job is claimed from the ring, not the wake key.

§3

The beat, and one blocker for a pool

consumer.ex / metronome.ex

A wake is the fast path, not the only path. When no wake arrives, BLPOP times out after one beat (:beat_ms, default 1000) and the loop runs its reap/promote pump regardless — so a due delayed job or an expired lease is recovered on the next beat even if nothing rang the doorbell. The wake makes a fresh enqueue prompt; the beat makes the time-driven work reliable.

A pool of standalone consumers would each hold their own BLPOP on the same wake key — a herd of blockers, one ring returning one of them. EchoMQ's opt-in EchoMQ.Metronome collapses that to one: a single process holds the one BLPOP emq:{q}:wake <beat> per queue and a registry of idle consumers, and on a wake it pokes each idle consumer to run EchoMQ.Lanes.claim/3 exactly once (one claim per idle consumer per wake). The herd is gone — one connection blocks — and readiness is fanned out fairly over BEAM messages. Select a fan size below.

One ring, the metronome's fan-out. Choose how many consumers are idle; the readout runs one wake.
metronome (1 block) one wake → one claim per idle consumer

Static verdict — the metronome holds one block per queue; a wake pokes each idle consumer once, so a job is served well before the beat.

Notes on ValkeyBLPOP serves the first element of a list and, when the list is empty, parks the client until another client pushes one; the engine returns exactly one of several clients blocked on the same key, in the order they parked (valkey.io/commands/blpop); LTRIM keeps the list within a fixed length (valkey.io/commands/ltrim).

door · the worked consumer & the loop in depth

codemojex rides the wake

EchoMQ.Consumer drains codemojex's guess queue through Lanes.claim, the player id arriving as the lane group; the consumer is returned the instant Codemojex.Game admits a guess JOB and rings the wake key. The full loop, the connector pool, and the metronome are the /echomq/queue Queue pillar — "door, not depth."

the pattern

Ring a signal when work is added, and the parked worker returns the instant it rings; one ring returns one parked worker.

its EchoMQ application

The admit Lua runs LPUSH emq:{queue}:wake '1' / LTRIM … 0 63; the consumer's BLPOP returns and drains. The beat is the fallback; one EchoMQ.Metronome blocks once for a pool. codemojex's guess queue rides it.

The wake key is a doorbell: one LPUSH returns one parked BLPOP, the beat catches the time-driven work, and one metronome blocks for a whole pool.

§4

Recap — a signal, not a queue

back to the hub

The consumer parks on BLPOP emq:{queue}:wake. The admit script rings the wake key with LPUSH '1' and bounds it with LTRIM 0 63; the parked block returns and the consumer drains the ring with Lanes.claim. The beat is the fallback for time-driven work, and one EchoMQ.Metronome holds the single block for a pool. That is the whole move blocking-vs-polling makes: park, ring, return — the spin gone.

§5

References

sources · related

Sources

Related in this course