B3.2 · dive 2 of 3
Jobs and lanes.
The bus moves branded entities. A job is a JOB id with a leased lifecycle; lanes drain them fairly; flows record one job waiting on another — relations again, not fields.
The idea
echo_mq · jobs.ex, lanes.ex, flows.exA job is a branded entity — a JOB id with a property bundle — and its life is a small, leased state machine. Jobs.enqueue writes a waiting job; Jobs.claim leases it to a worker and makes it active; Jobs.complete finishes it, Jobs.retry returns it to waiting with a backoff, and a job that exhausts its tries fails. The lease is what makes the bus safe: a worker that dies holding a job loses the lease, and the job is reclaimable rather than lost.
A queue is not one line; it is a set of lanes, each named by a branded id, drained fairly. Lanes.claim is how every consumer takes work, and it spreads claims across lanes so no one lane starves the rest — the fair-lanes property. Group-aware pause and resume act on a lane without touching the queue, so one tenant can be held while the others run. Work that should ride a lane and is enqueued ungrouped is never claimed; the lane is the unit of fairness and of control.
Jobs depend on jobs, and a dependency is a relation, not a field. Flows.add records that a parent waits on its children; when the last child completes, the parent is released, its children’s values readable from the parent. A flow is a small graph of branded ids with its own rules — fan-in, failure policy, grandchildren — owned by the bus, the same move B2 made for edges: the relation is a system, referenced by id, not embedded in either job.
The doors
the sibling coursesEchoMQ — the protocol, in depth
The bus B3 narrates, taught rung by rung: the keyspace, the Lua inventory, and the 3.0 Stream Tier — XADD, XREADGROUP, XACK — with conformance on Valkey.
Redis Patterns Applied
The substrate patterns under the bus: sorted sets, atomic Lua moves, locks, and streams — the judgement layer.
/echo-persistenceEcho Persistence — the durable floor
Beneath the volatile tiers: the Graft page engine built twice, the commit-LSN loop, and the stream archive on Tigris. Where Part I says state lives somewhere, this says where it lives durably.
References
sources · relatedSources
- Oban — Robust job processing in Elixirthe Postgres-backed prior art for jobs, queues, and workflows
- Valkey — Sorted Setsthe scored structure a scheduler and the lanes are built on
- Lamport — Time, Clocks, and the Ordering of Events (1978)the happens-before a lease and a queue order by
Related
- BCS — the course homethe law, the id anatomy, the chapter map
- Echo Persistencethe durable floor beneath the bus