B3.3 · dive 3 of 3
The Stream Tier.
EchoMQ’s Stream Tier sits beside the queue. A writer appends to a log, consumer groups read and acknowledge, and retention is a policy a system sets.
The idea
echo_mq · stream.ex, stream_consumer.exEchoMQ’s Stream Tier sits beside the job queue. A queue is for work that is claimed and completed once; a stream is an append-only log that many readers consume at their own pace. The tier is built additively: the writer, the readers, and retention each shipped as a rung with its own conformance probe, and the stream keys are born braced and branded like every other key on the bus.
EchoMQ.Stream is the writer — it appends an entry to the log and returns its id, the log ordered by append the way the property store is ordered by mint. EchoMQ.StreamConsumer is the reader law: a consumer group reads new entries, acknowledges them, and resumes where it left off, so a reader that restarts does not replay what it has already handled. A deliberate polyglot seam keeps the read side reachable from any language that speaks the wire, not only the BEAM.
A log that only grows is a leak, so retention is a policy, not a default. EchoMQ.Stream.trim bounds the log by length or age, and the trimming driver is named and opt-in — a queue keeps its history until a system decides how much to keep. What is trimmed is not lost: EchoStore.StreamArchive folds trimmed segments into the durable Graft floor — deep history without resident memory, readable beside the live tail. The stream is the same idea as the store seen sideways: identity orders the entries, the boundary owns them, and what survives is a choice the system makes, not an accident of volume.
The doors
the sibling coursesEchoMQ — the protocol, in depth
The bus B3 narrates, taught rung by rung: the keyspace, the Lua inventory, and the 3.0 Stream Tier — XADD, XREADGROUP, XACK — with conformance on Valkey.
Redis Patterns Applied
The substrate patterns under the bus: sorted sets, atomic Lua moves, locks, and streams — the judgement layer.
/echo-persistenceEcho Persistence — the durable floor
Beneath the volatile tiers: the Graft page engine built twice, the commit-LSN loop, and the stream archive on Tigris. Where Part I says state lives somewhere, this says where it lives durably.
References
sources · relatedSources
- Valkey — Introduction to Streamsthe append-only log and consumer groups the tier is built on
- Kreps — The Log: What every software engineer should knowthe log as the shared abstraction beneath a stream
- Tigris — Conditional operationsthe create-only fence an archived stream lands behind
Related
- BCS — the course homethe law, the id anatomy, the chapter map
- Echo Persistencethe durable floor beneath the bus