R5.01.2 · dive 2

Replay and rebuild.

State is derived, not stored: read the log in order and fold the events into an accumulator to reconstruct the present. XRANGE is the read; the fold is the application of each entry.

XRANGE key - + returns every entry of a stream from the first (-) to the last (+) in id order, and a bounded slice is XRANGE key <start> <end> COUNT n. Reading in order is the precondition for the rebuild: the fold applies events one at a time, and the result is correct only if they arrive in the order they were appended. Move the range bounds; the readout shows which entries are returned.

XRANGE reads in order State is a fold Immutable events In EchoMQ References

XRANGE returns entries in id order, within the bounds. Select a range; the readout reports which entries it returns.

§1

XRANGE reads the log in order

replay-and-rebuild.md · the read

XRANGE key - + returns every entry of a stream from the first to the last, in id order. - is the smallest possible id and + the largest, so the pair reads the whole log; a bounded slice is XRANGE key <start> <end> COUNT n. The entries come back as [id, [field, value, …]] pairs, already sorted by id — which, because the log is append-only, is the order they happened in.

XRANGE orders - +                    # the whole log, oldest first
# 1) …000-0  type placed  amount 4200
# 2) …001-0  type filled  amount 4200
# 3) …002-0  type closed

Reading in order is the precondition for the rebuild: the fold applies events one at a time, and the result is correct only if they arrive in the order they were appended. XRANGE guarantees that order; no sort is needed afterward.

XRANGE - + reads the whole log in id order, oldest first — the order the entries were appended, which is exactly what the fold needs.

§2

State is the fold of the log

replay-and-rebuild.md · the rebuild

Rebuilding state is a left fold: start with an empty accumulator, and for each entry apply a transition function that takes the accumulator and the event and returns the next accumulator. The current state is the accumulator after the last event. The same log folded the same way always yields the same state — the rebuild is deterministic in the log.

state = {}                              # the empty accumulator
for entry in XRANGE orders - +:        # in id order, oldest first
    state = apply(state, entry)         # place -> open, fill -> filled, close -> closed
# state is now the present, reconstructed from the events

The fold applies each entry to the accumulator, in order. Step to an entry; the accumulator is the state after applying it.

State is the left fold of the log: apply each entry to an accumulator in order, and the result after the last entry is the present — recomputed, never read from a stored "current state".

§3

The events are immutable; the fold is replaceable

replay-and-rebuild.md · fixed history · the snapshot

An event, once appended, is never edited. A correction is a new event appended to the end, not an edit of an old one. So the log is fixed history, and the fold over it is reproducible: replay the same entries and get the same state. When the way state is derived has a bug, the fix is to change the fold and replay — the events stay exactly as recorded, and the new state is computed from the unchanged history.

A long log is work to fold, so a rebuild is bounded by a snapshot: the folded state saved at a known id, plus the events after it. A reader loads the snapshot and folds only the tail. The snapshot is an optimisation of the replay, never a replacement for the log — the log is still the source of truth.

Notes on ValkeyXLEN key returns the number of entries in a stream — the size of the log to fold, and the count a snapshot bounds the replay against — valkey.io/commands/xlen.

The events are fixed; the fold is replaceable. Fix the derivation and replay, and a snapshot plus the tail bounds the work — the log stays the source of truth.

§4

In EchoMQ — EchoMQ.Stream.read/6

echo_mq/stream.ex · read/6

EchoMQ.Stream.read(conn, queue, name, from, to, count) is the read side of EchoMQ's Stream Tier, in real code in echo/apps/echo_mq. It wraps XRANGE and parses each entry back into a {branded, fields_map} tuple in mint order:

# EchoMQ.Stream.read/6 — XRANGE, parsed to {branded, fields} in mint order (real)
EchoMQ.Stream.read(conn, "orders", "events")          # from "-", to "+", whole log
#  XRANGE emq:{orders}:stream:events - +
#  -> {:ok, [
#       {"EVT0Nk2…", %{"type" => "placed", "amount" => "4200"}},
#       {"EVT0Nk2…", %{"type" => "filled", "amount" => "4200"}},
#       {"EVT0Nk2…", %{"type" => "closed"}}
#     ]}                                          # mint order, ready to fold

The branded record id is recovered from the stored id field; the remaining pairs are the payload as a map. The entries arrive in mint order — which equals the XRANGE id order, because the writer mints monotone ids — so a fold over the returned list reconstructs the current state with no extra sort. from and to default to the full range -/+, and a COUNT bounds the slice for a paged replay. The read is un-grouped: it is the order-theorem proof surface, not the consumer group (that is R5.02).

the pattern

Read the log in order and fold the events into an accumulator to reconstruct current state; the events are immutable, the state is derived.

its EchoMQ application

EchoMQ.Stream.read/6 issues XRANGE emq:{q}:stream:<name> and hands back {branded, fields_map} tuples in mint order, ready to fold; the entries are never mutated, so the fold is reproducible.

EchoMQ's read is the rebuild's read: XRANGE parsed to mint-ordered tuples a fold turns back into the present.

/echomq/bus

A door, not a depth

The consumer-group read that lets many readers fold the same log at their own pace, the acknowledgement that resumes a reader where it left off, and the snapshot-and-replay of a long history are the subject of the dedicated EchoMQ course. From here, this dive opens onto the Bus pillar — the reader law in depth. This dive teaches the fold; that course teaches the reader.

§5

References

sources · related

Sources

Valkey — XRANGE — read a range of stream entries in id order, the rebuild read.
Valkey — Introduction to Streams — the -/+ range bounds and the entry shape a reader folds.
Valkey — XLEN — the length of a stream, the size of the log to be folded.
Kreps, J. — The Log — replaying the log as the way state is derived.

Related in this course

R5.01 · Event sourcing on Streamsthe module hub: state is the replay of the log
R5.01.1 · The append-only logthe write side, XADD
R5.01.3 · The cursorthe entry id as resume position and time bound
R5.02 · Stream consumer patternsreading the log reliably with groups
/echomq · Busthe reader law in depth