jonnify·bcs /bcs/bus/the-stream-tier

B3.3 · dive 3 of 3

The Stream Tier.

EchoMQ’s Stream Tier sits beside the queue. A writer appends to a log, consumer groups read and acknowledge, and retention is a policy a system sets.

An append-only log: written, read, trimmed. Select a part.
the stream — an append-only log trimXTRIM — trim(maxlen) is a named, opt-in policy EchoMQ.Stream.addXADD StreamConsumerany language · RESPXREADGROUP · XACK — a group resumes where it left off
readout · Select a part. The Stream Tier is an append-only log beside the queue: a writer appends, consumer groups read and ack, retention trims.
§1

The idea

echo_mq · stream.ex, stream_consumer.ex

EchoMQ’s Stream Tier sits beside the job queue. A queue is for work that is claimed and completed once; a stream is an append-only log that many readers consume at their own pace. The tier is built additively: the writer, the readers, and retention each shipped as a rung with its own conformance probe, and the stream keys are born braced and branded like every other key on the bus.

EchoMQ.Stream is the writer — it appends an entry to the log and returns its id, the log ordered by append the way the property store is ordered by mint. EchoMQ.StreamConsumer is the reader law: a consumer group reads new entries, acknowledges them, and resumes where it left off, so a reader that restarts does not replay what it has already handled. A deliberate polyglot seam keeps the read side reachable from any language that speaks the wire, not only the BEAM.

A log that only grows is a leak, so retention is a policy, not a default. EchoMQ.Stream.trim bounds the log by length or age, and the trimming driver is named and opt-in — a queue keeps its history until a system decides how much to keep. What is trimmed is not lost: EchoStore.StreamArchive folds trimmed segments into the durable Graft floor — deep history without resident memory, readable beside the live tail. The stream is the same idea as the store seen sideways: identity orders the entries, the boundary owns them, and what survives is a choice the system makes, not an accident of volume.

§4

The doors

the sibling courses
§2

References

sources · related

Sources

Related