jonnify·echd /echd/bus/streams

echd — Echo on Encore · E3.2

Streams Supersede PubSub

A topic is a log and a subscription is a group — so a message is retained, acknowledged, replayable, and readable from any language, where PubSub delivers once and forgets.

Vision · real substrate

echd speaks in the present tense over a build ladder. Every module, verb, key, and port named on this page is real; the composition into one standing platform is the vision the ladder builds.

§1

The module

docs/echo_mq/emq.streams.md:29-42

Encore's PubSub is fire-and-forget: a topic delivers to whoever is subscribed at that instant, and a subscriber that was down misses the message forever. The stream tier replaces it with something stronger, and owns the difference. A topic becomes a Valkey stream — an append-only, mint-ordered log — and a subscription becomes a consumer group. A message is not gone the moment it is delivered; it is retained, acknowledged, and replayable.

Turn PubSub into a stream and three properties arrive together. A consumer group lets many workers share one topic without seeing each other's entries — XREADGROUP distributes, XACK completes. At-least-once falls out of the pending-entries list: a delivered entry stays pending until acknowledged, and XAUTOCLAIM rescues the work of a consumer that died. And because the group is a Valkey construct rather than a library, the seam is polyglot — a non-BEAM reader stands in the same group as the BEAM. Retained, replayable, at-least-once, any language: the four things PubSub is not.

§2

The stream tier, drawn

hover a facet
A topic is a log, a subscription is a group · the stream tier · hover to read
the topic — a Valkey stream (XADD, mint-ordered) XADD → consumer group — XREADGROUP fans entries to consumers · XACK done at-least-once — the pending list + XAUTOCLAIM reclaim · XTRIM retention one group, many runtimes — BEAM · Go · Python share the delivery a topic is a log; a subscription is a group — retained, replayable, at-least-once
Consumer groups — the subscription, owned. A topic is a stream and a subscription is a consumer group: XREADGROUP hands each new entry to exactly one consumer in the group, so N workers share the load without seeing each other's entries, and XACK marks an entry done. Add a consumer and throughput scales.
At-least-once — nothing dropped on the floor. Every delivered-but-unacked entry sits in the pending-entries list; if a consumer dies, XAUTOCLAIM hands its work to a living one after an idle timeout. Retention is policy — XTRIM bounds the log by length or minimum id — and a reader replays from any id. PubSub forgets; the stream tier remembers.
The polyglot seam — one group, any language. Because the group lives in Valkey, not in a runtime, a Go consumer and a BEAM consumer join the same group and partition the same stream. PubSub cannot fan out durably across languages; the stream tier does it by construction.

Hover or focus a facet to read it; click to pin. Without scripting, all three read in sequence.

§3

The dives

three moves down
D1

Consumer Groups

A subscription is a group — XREADGROUP partitions the log, XACK completes, the group tracks who has what.

built D2

At-Least-Once

The pending-entries list, XAUTOCLAIM for the dead, XTRIM retention, replay from any id — PubSub forgets, this remembers.

built D3

The Polyglot Seam

One group, any language — why a Go consumer and a BEAM node share a stream PubSub could never fan out.

built

← Back to modules

§4

Doors

the tier in depth
/echomq

EchoMQ — the protocol, in depth

The stream tier verb by verb: XADD, XREADGROUP, XACK, XAUTOCLAIM, and the retention and replay commands.

the log, the group, the pending list — the tier this module compresses.

 /redis-patterns

Redis Patterns, Applied

Streams as a pattern: consumer groups, at-least-once delivery, and the reliable fan-out PubSub cannot give.

groups over pub/sub — when durability and replay decide the choice.
§5

References

sources · related

Sources

  • Valkey — Streamsthe log, consumer groups, XACK, and the pending-entries list the stream tier rides
  • Valkey — Pub/Subthe fire-and-forget primitive the stream tier supersedes — no retention, no acks, no replay

Related