jonnify·redis /redis-patterns/streams-events/custom-events

R5.04 · Streams & Events — module hub

Custom events & projections.

Give every kind of domain event its own named stream, append each as an immutable entry, and fold a window of those entries into a projection — a read-model shaped for one question.

This is event sourcing (R5.01) applied to user-defined events: the same append-only log, but the event types and the read-models are yours to choose. The grounding is EchoMQ's real Stream Tier (echo/apps/echo_mq): EchoMQ.Stream.append(conn, queue, name, fields) takes a caller-chosen name and issues XADD emq:{q}:stream:<name>, so the name is the domain-event channel; read_window/6 and read_since/5 read a closed or half-open time window to fold. The three dives take it apart — the channel, the projection, the reserved-name discipline.

A named stream is a domain-event channel — one log per type. Select an event type, then read which entries its stream holds and who consumes it.
three named streams · one log per event type room-opened room ROM… room ROM… room ROM… guess-scored total 480 total 320 total 600 game-settled winner PLR… pot 60
readout · Select an event type. Each domain-event type has its own named stream — one append-only log per channel, read by exactly the readers that care about it.
§1

The pattern in one move

custom-events.md · R5.01 reapplied

R5.01 taught the log as the source of truth and the fold as the rebuild. Custom events take that one step further: the events are not a fixed catalogue (placed, filled, closed), they are whatever the domain produces — a room opened, a guess scored, a game settled. Each event type gets its own named stream, so one log per channel keeps a reader from sifting unrelated entries. And a projection is a fold scoped to a slice of the log: take the entries in a time window and reduce them to exactly the shape one view needs — a per-room count, an hourly total, a running tally — recomputed from the log, never a separate table kept in sync.

Two moves carry the module, and they are two of the three dives. A custom event is a named stream: XADD <stream-name> … appends to a stream chosen by name, and the name is the domain-event channel. A projection is a windowed fold: read the slice of a stream between two instants and fold it to a read-model. The third dive is the discipline that keeps custom names safe — a stream's id field and a couple of key suffixes are reserved, and a custom event keeps clear of them.

Custom events change what the events are and which read-models fold them — one named stream per event type, one windowed fold per question — on the same append-only log R5.01 taught.

§2

A custom event is a named stream

custom-events.md · the name is the channel

A stream is identified by its key. Give each domain-event type its own key, and one append-only log per type follows: room-opened events go to one stream, guess-scored to another, game-settled to a third. A reader of the settlement feed reads only settlement entries; it does not page through guesses to find them. Adding a new event type is adding a new name — no schema migration, no shared table.

XADD room-opened   * room ROM0Nk2…  set animals  fee 1     # -> "…000-0"
XADD guess-scored  * game GAM0Nk3…  player PLR0…  total 480 # -> "…000-0" (a separate log)
XADD game-settled  * game GAM0Nk3…  winner PLR0…  pot 60    # -> "…000-0"

Each entry is immutable once written and carries a time-ordered id. The three streams are independent logs over the same instance; a projection reads one of them, or fans across several, and folds. The cost of one stream per type is a key per type; the buy is a focused log per feed.

The key is the channel: one named stream per domain-event type, each a focused append-only log, a new type costing only a new name.

§3

A projection is a windowed fold

custom-events.md · time is an id range

A projection answers one question from the log: how many rooms opened this hour, what each player's running total is, which games settled today. It is a fold — start with an empty accumulator, apply each entry in order — scoped to a window of the log. The window is expressed in time, and time is an id range, because a stream entry id begins with the millisecond the entry was added.

# the events of one hour, folded to a per-room count
XRANGE room-opened 1719312000000-0 1719315600000-0   # a closed [t0, t1] window
# everything from an instant onward, folded to a running total
XRANGE guess-scored 1719312000000-0 +                # a half-open [t0, ∞) window
A projection is the fold of one window — the entries inside it reduce to a read-model. Select a window over the guess-scored log; the readout reports what folds and the total it makes.
window 200t₀ 480t₁ 320t₂ 600t₃ projection—
readout · Select a window. The projection is the fold of the entries inside it — here a sum of the total field, recomputed from the log, never a table kept in sync.

Notes on ValkeyA bare <ms> bound is treated as <ms>-0, so a millisecond instant becomes the floor id of a windowed read, and a range between two instants is the closed window a projection folds — valkey.io/commands/xrange.

A projection is the deterministic fold of a time window of the log; many projections fold the same log into different shapes, each recomputed and never stale.

§4

Reserved names — what a custom event must avoid

custom-events.md · the reserved few

Custom names are free, but a few are spoken for. The stream stores its canonical record id under a field named id, so a payload must not also name a field id — the reader would not be able to tell the stored id from the payload field. A key suffix is reserved for the archive watermark, so a stream's own name must not collide with it. And the whole keyspace is namespaced under a reserved prefix, so a custom stream lives inside that namespace rather than beside it.

The discipline is small and mechanical: keep custom event names and payload field names clear of the reserved few, and every other name is yours. The third dive, reserved-name discipline, takes it apart against the real EchoMQ surfaces.

Three names are reserved — the stored-id field, an archive suffix, a namespace prefix; keep custom names clear of those few and the rest is the domain's to choose.

§5

When to use it, when to avoid it

custom-events.md · the trade

Reach for custom events and projections when the domain produces many kinds of events and several views each need a different slice or shape of them: an activity feed, per-entity counters, time-windowed rollups, an audit that other readers also consume. One log per event type keeps each feed focused, and a projection per question keeps each read-model recomputable from the log.

Avoid a stream per event type when the events are few and one combined log reads fine — splitting then adds keys without buying focus. And a projection over a long window is work: where a rollup is read far more often than the window changes, fold once and cache the result, or trim the log to bound the replay (R5.02). A projection is recomputable, which is the point; recomputing it on every read when the inputs rarely change is waste.

Split the log by event type and project per question when many views fold many event kinds; keep one combined log and a cached rollup when the events are few or the projection is stable.

§6

Applied — EchoMQ's caller-chosen name and windowed read

echo_mq/stream.ex · append/4 · read_window/6

EchoMQ's Stream Tier appends to a stream chosen by the caller. EchoMQ.Stream.append(conn, queue, name, fields) takes a name, mints an EVT-branded record id host-side, and issues XADD emq:{q}:stream:<name> <xadd_id> id <branded> <fields…> — so the name is the domain-event channel and the key is emq:{q}:stream:<name> via EchoMQ.Keyspace.queue_key/2. One brand per stream (EVT, the kind door in EchoMQ.Stream.Id) keeps the entry order, the id sort, and the mint order one and the same.

# EchoMQ.Stream.append/4 — the name IS the domain-event channel (real)
EchoMQ.Stream.append(conn, "cm", "room-opened",  [{"room", room}, {"set", "animals"}])
EchoMQ.Stream.append(conn, "cm", "guess-scored", [{"game", game}, {"player", player}, {"total", "480"}])
EchoMQ.Stream.append(conn, "cm", "game-settled", [{"game", game}, {"winner", winner}, {"pot", "60"}])
#  -> XADD emq:{cm}:stream:room-opened  <xadd_id> id <branded> room … set animals
#  -> {:ok, "EVT0Nk2…"}    (the branded id is the receipt)

A projection is a windowed read folded to a read-model. EchoMQ.Stream.read_window(conn, queue, name, t0, t1, count) reads the closed [t0, t1] window, and EchoMQ.Stream.read_since(conn, queue, name, t0, count) the half-open [t0, ∞) window. Both derive the bounds host-side — minid_floor/1 turns t0 into "<ms>-0", maxid_ceil/1 turns t1 into "<ms>-<max-seq>" — so the wire only ever sees an <ms>-<seq> id and the window edges are exact. The returned {branded, fields_map} tuples are in mint order; fold them to the projection.

The worked consumer is codemojex — a Telegram emoji-guessing game on this stack (echo/apps/codemojex). Codemojex.Rooms.create_room/3 opens a room (a ROM), the first joiner starts a game (a GAM), Codemojex.Guesses.submit/3 enqueues a guess as a JOB on the player's PLR lane, and Codemojex.Scoring.score/2 scores it. Each of those lifecycle moments is a domain event appended to its own named stream; a windowed read of guess-scored folds to the live activity feed, and a read of game-settled folds to the day's results.

Notes on ValkeyA stream key carries the queue name inside the braces (emq:{q}:stream:<name>), so every named stream of one queue hashes to one of the 16384 cluster slots and a multi-key script stays legal — valkey.io/topics/cluster-spec.

the pattern

Give each domain-event type its own named stream and fold a time window of it into a projection — a read-model shaped for one question, recomputed from the log.

its EchoMQ application

EchoMQ.Stream.append/4 takes a caller-chosen name → XADD emq:{q}:stream:<name>; read_window/6 and read_since/5 read a closed [t0, t1] or half-open [t0, ∞) window in mint order to fold. The worked consumer is codemojex's room / guess / settle events.

Redis offers a named append-only log and an id-ranged read; EchoMQ's Stream Tier makes the name the domain-event channel and the window an exact mint-time range — the projection is the fold over it.

/echomq/bus

A door, not a depth — the EchoMQ Bus pillar

This module cites the append with a caller-chosen name and the windowed reads as proof the named log and the time-bounded read ship. The consumer-group reader, the polyglot seam, retention, and the archive fold are the subject of the dedicated EchoMQ course. Open onto the Bus pillar — the broadcast and the retained, replayable log.

/echo-persistence

Where a windowed history ages out

A log is trimmed under a retention policy, and the trimmed slice folds into the durable page tier and, beyond it, to remote object storage behind a create-only commit fence. That floor — the durability dial a system turns — is the subject of the EchoStore persistence course. The figures here are drawn from the Branded Component System bus chapter, /bcs/bus.

After R5.04, the Streams & Events chapter closes with a workshop — a live codemojex activity feed off the stream lifecycle. Return to the Streams & Events chapter to continue.

§7

The three dives

channel → projection → discipline

Each dive takes one move, in order: the named stream as a domain-event channel, the windowed fold as a projection, and the reserved-name discipline that keeps custom names safe. Read them in order — the channel first, the projection second, the discipline third.

The three dives ground in the same writer and reader: the named append, the windowed read, and the reserved names. Begin with domain events on the stream.

§8

References

sources · related

Sources

Related in this course