← the depth behind redis-patterns R5 · Streams & Events
The Bus · module 03
The consumer group
Many readers, one log — each entry handed to one, acknowledged, and resumable across a crash.
Module 02's read/6 is a stateless replay: every reader
sees the whole log from the head. A consumer group is the other kind of read — the
reliable, distributed one. Many cooperating readers share one stream, each entry is
delivered to one of them, acknowledged with XACK, and resumable. The group
remembers each consumer's un-acked backlog — the PEL — so a reader that crashes
resumes where it left off without replaying what it already handled. The promise in one line:
read >, ack, resume. And the cost, named up front:
at-least-once, not exactly-once. It is all real code.
Each entry once, resumable
A group is one stream read by many cooperating consumers, where the group keeps each consumer's PEL — the list of entries delivered to it but not yet acked. That one piece of bookkeeping is what makes the read reliable: an entry goes to one consumer, the consumer settles it, and what it does not ack survives to be redelivered. Pick a state below — the steady fan-out, a consumer crash that leaves a PEL, the peer reclaim — and read what the group does, the Valkey verb beneath it, and the guarantee.
A group over one stream · pick a state
The PEL is the whole trick: an entry goes to one consumer, and what is not acked is not lost —
it waits, redeliverable, until someone settles it. Read >, ack, resume.
The dives
Three dives, each grounding in the real echo/apps/echo_mq code — the lazy group
door, the two recovery mechanisms (self then peers), and the at-least-once handler with the idempotency cost it
carries. The group verbs are issued direct — there is no Lua here.
The group door
XGROUP CREATE … MKSTREAM on start: swallow only BUSYGROUP, everything else loud, and the declared :group_start with no default.
Recover self, then peers
Drain the PEL on (re)start to recover self; the XAUTOCLAIM beat to recover dead peers; the blocking > read on a private lane.
At-least-once & the handler
The exact-mirror handler: :ok acks, an error leaves it un-acked; attempts is the delivery-count; the re-claim ordering cost, and idempotency as its price.
Redis Patterns Applied
This module is the depth behind the Redis Patterns Applied chapter that doors here: R5 · Streams & Events — the consumer group, the pending-entries list, and the at-least-once delivery the pattern names, made concrete on the wire. There the pattern is the door; here is the law.
The rest of the pillar
The consumer group is the reliable read. It sits on the stream log (module 02), the writer that mints the entries it consumes. Beyond it, time-travel reads the log by a mint instant, and retention bounds it — folding what it trims into the durable floor. The events channel beside them is the present-tense surface the log makes durable.
The Bus
The pillar landing: the two surfaces, one wire — events and the stream log.
The stream log
The writer this group reads: the host-side mint, the order theorem, the claims-only id field.
Time-travel
Read the log by a mint instant: a DateTime becomes an id range bound for backtest, audit, and debug.
Echo Persistence
The durable floor a trimmed stream history folds into — the archive frontier.
The events channel and the stream log are ready to read; time-travel and retention are the Bus's remaining modules, each added as its pages land. This module is the reliable read over the writer they share.
References
Sources
- Valkey — Introduction to Streams — consumer groups, the PEL, and at-least-once delivery.
- Valkey — XREADGROUP — the group read;
>for new entries,0for the PEL. - Valkey — XACK — the acknowledgement that retires an entry from the PEL.
- Valkey — Cluster specification — the
{q}hashtag co-locates a queue and its stream on one of 16384 slots.
Related in this course
- The group door — ensuring the group on start.
- Recover self, then peers — the two recovery mechanisms.
- At-least-once & the handler — the handler and the idempotency cost.
- The stream log — the writer this group reads.
- The Branded Component System · the Bus — the manuscript chapter this module realizes.