R5.01.1 · dive 1
The append-only log.
The log is the source of truth: every state change is appended, never updated in place. A Redis
Stream is the structure — XADD adds an entry to the end and returns its id.
A Redis Stream grows one way: XADD appends an entry to the end and returns its
time-ordered id. There is no in-place update of a written entry — the log only grows forward, and a later
append always carries a larger id. That single property, append never overwrite, is what makes it the right
structure for event sourcing. Append placed, filled, closed; the readout
shows the growing log and its ids.
A stream is an append-only log
the-append-only-log.md · the structureA Redis Stream is an ordered sequence of entries, each a small set of field-value pairs under an id. The only way
to add to it is XADD, which appends to the end and returns the new entry's id. There is no in-place
update of a written entry: a stream grows forward only. That single property — append, never overwrite
— is what makes it the right structure for event sourcing.
XADD orders * type placed amount 4200 # append; * mints a time-ordered id -> "…000-0" XADD orders * type filled amount 4200 # append the next event -> "…001-0"
The id is <milliseconds>-<sequence>: the millisecond the entry was added, then a sequence
number that breaks ties within the same millisecond. A later append always carries a larger id than an earlier
one, so the id orders the log. The * asks the server to mint the id; an explicit id may be supplied
instead, and must exceed the current top of the stream or XADD rejects it.
A stream is an append-only sequence: XADD adds at the end under a rising id, nothing is
overwritten, and the id orders the log.
Every transition is an event
the-append-only-log.md · the historyEvent sourcing records each change as its own entry. An order placed, an order filled, an order cancelled — three events, appended in the order they happened, never collapsed into one mutable "order" row. The log holds the whole history; the present is whatever the history adds up to. Because nothing is overwritten, the log is an audit trail for free: it says not only what is true now but how it came to be true.
Each state change is its own appended entry, so the log is the whole history — an audit trail for free, not a single mutable value.
The append rejects a smaller id
echo_mq/stream.ex · the order keptA stream's entry id must strictly exceed its current top. XADD with an explicit id at or below the
top is refused — the server answers an error rather than break the order. This is the property that keeps
the log a log: the id sequence is strictly increasing, so reading by id reads in append order. A writer that mints
ids in order never trips it; one that does trips it loudly, and the error is not swallowed.
<ms>-<seq>, and it must exceed the top.
Select an append against a stream whose top is …001-0; read whether it is accepted.The id must strictly exceed the top, so the sequence is strictly increasing and reading by id reads in append order; an equal or smaller id is refused, not silently dropped.
In EchoMQ — EchoMQ.Stream.append/4
echo_mq/stream.ex · append/4EchoMQ.Stream is the writer of EchoMQ's Stream Tier.
EchoMQ.Stream.append(conn, queue, name, fields) does the append in four steps, in real code in
echo/apps/echo_mq: it mints an EVT-branded record id host-side, derives the explicit
XADD id from it, issues XADD emq:{q}:stream:<name> <xadd_id> id <branded>
<fields…>, and returns {:ok, branded} — the branded id is the receipt.
# EchoMQ.Stream.append/4 — the append, shaped to the wire (real) EchoMQ.Stream.append(conn, "orders", "events", [{"type", "placed"}, {"amount", "4200"}]) # 1. mint an EVT-branded record id host-side (EchoData.Snowflake.next_branded("EVT")) # 2. derive the explicit XADD id from it (Stream.Id.xadd_id/1 — "<ms>-<tail22>") # 3. XADD emq:{orders}:stream:events <xadd_id> id <branded> type placed amount 4200 # 4. -> {:ok, branded} (the branded id is the receipt)
The writer owns the mint, so there is nothing to spoof — the id comes from the host, not a client. The
14-byte branded id is stored as the stream entry's id field, so a reader in any language gets the
canonical id without re-encoding it. The append returns {:ok, branded}; a Valkey rejection of an id
at or below the top is mapped to {:error, :nonmonotonic} — never swallowed, never retried with
*, because that error is the wire telling the truth that an upstream mint-order violation happened.
The order theorem holds because a single writer per stream mints strictly increasing snowflakes: the next id
always exceeds the stream top, so no XADD rejection is possible. Stream order equals id sort equals
mint order — the log is ordered by append the way the property store is ordered by mint.
Notes on ValkeyXADD key * field value appends an entry and
returns its <ms>-<seq> id; an explicit id at or below the stream top is refused with an
error rather than reordered —
valkey.io/commands/xadd.
Append every state change as an immutable entry to a log; the id orders it, and a smaller id is refused so the sequence only grows.
EchoMQ.Stream.append/4 mints an EVT id, derives the XADD id, issues
XADD emq:{q}:stream:<name>, and returns the branded id; an id≤top rejection
surfaces as {:error, :nonmonotonic}.
EchoMQ's append is the pattern in real code: mint an ordered id, append the entry, return the receipt, and surface a mint-order violation instead of breaking the log.
A door, not a depth
The batch append, the explicit kind door on a caller-supplied id, and the way the writer's monotone mint proves the order theorem are the subject of the dedicated EchoMQ course. From here, this dive opens onto the Bus pillar — the writer and the retained, replayable log in depth. This dive teaches the append; that course teaches the writer that runs it.
References
sources · relatedSources
- Valkey — XADD — append an entry to a stream, assigning a time-ordered id; the explicit-id rejection of an id at or below the top.
- Valkey — Introduction to Streams — the append-only log and the
<ms>-<seq>entry id. - Valkey — Cluster specification — the
{q}hash tag forcing a queue's stream key onto one of the 16384 slots. - Kreps, J. — The Log — the append-only log as the shared abstraction.
Related in this course
- R5.01 · Event sourcing on Streamsthe module hub: the log is the source of truth
- R5.01.2 · Replay and rebuildread the log back and fold it to state
- R5.01.3 · The cursorthe entry id as resume position and time bound
- R3.01 · Processing listthe queue beside the log, claimed once
- /echomq · Busthe writer in depth