The Queue · module 04
Lifecycle controls
Enqueue, claim, complete is the spine. The controls are everything an operator and a worker reach for around it.
Beyond the state machine sit the controls over its edges: over time — schedule a
job for later, retry it on a curve, run it on a cadence; over the worker in hand — cancel a long
handler cooperatively, checkpoint its lease so it is not reaped mid-work, recover the ones that stalled; and over
the whole queue — pause claiming, drain the backlog, obliterate a paused queue, and reach into one
job's row to fix it. Each is a real transition in echo/apps/echo_mq — a score, a pure
function, a fenced script. Schedule, promote, delay, extend, the stall sweep and reprocess have native twins on a
module-loaded node (and two native verbs, EMQ.RESCHEDULE and
EMQ.REQUEUE, have no Elixir twin); repeat, cancel and the admin verbs run only as scripts
or on the host. Pick a control below, then take the dives.
Three scopes of control
The controls divide into three scopes — time, the worker in hand, and the whole queue. Pick a control to read
the real module function it names, the structure it touches, the one-line effect, and the native command sent
for it on a module-loaded node, where one exists. Over the default wire 4 the module functions evaluate their
scripts; over a wire: 6 connector each one with a native twin sends that command
instead, and a script without one is refused with {:wire6_unsupported, name}. The map
is the module.
The control map · pick a control
touches: the schedule set emq:{q}:schedule
effect: park a job at a run-at score, a visibility fence out of pending, not a second queue; promote releases the due ones on the server clock.
native: EMQ.SCHEDULE · EMQ.PROMOTE; EMQ.RESCHEDULE rescores a scheduled job and has no Elixir twin
Every control is a real function over the owned keyspace — a score moved, a curve computed, a field set. Learn the map and you have read the control plane.
The dives
Three dives read the control plane in depth, each grounded in real echo/apps/echo_mq
code: control over time, over the worker in hand, and over the whole queue.
Scheduling & recurrence
enqueue_at/6 and enqueue_in/6 park a job on the schedule set at a run-at score; promote releases the due ones; delay sends a job in hand back; Backoff is the pure curve; Repeat is the cadence on the host clock. Native twins: EMQ.SCHEDULE, EMQ.PROMOTE, EMQ.DELAY.
Cancellation & checkpoints
The cooperative cancellation token and request/4; extend_lock checkpoints the lease and answers :not_active once it is reaped; the count-thresholded stalled sweep recovers or dead-letters. Native twins: EMQ.EXTEND, EMQ.STALLED.CHECK.
The operator plane
Admin pause / resume / drain / obliterate over the whole queue, and the per-job verbs on Jobs — update the payload, write progress, append logs, remove, reprocess. Native twin: EMQ.REPROCESS.
Pattern & implementation
Where Redis Patterns Applied teaches the delay, schedule, and priority family — a ZSET scored by run-at time, a promote pass, a backoff curve — this module is where that family lands as a control plane you own.
Delay, schedule, and priority over Redis: a sorted set scored by run-at time, swept by a promote pass, with a retry curve computed client-side. Time, delay & priority teaches the family.
The same machine, owned: @schedule parks on the schedule set,
@promote releases the due ones on the server clock, Backoff is
the curve a caller computes, and EchoMQ.Admin is the operator plane above it.
Take the dives in order, or jump to the operator plane if you are here for the runbook. The control plane sits on the protocol — the keyspace and the Lua layer every control runs on.
References
Sources
- Valkey — ZADD — the schedule-set insertion at the run-at score.
- Valkey — ZRANGEBYSCORE — the due read the promote pass runs.
- Redis — EVALSHA — the load-once dispatch every control verb runs by SHA.
- Valkey — Documentation — the substrate of record EchoMQ is backed by.
Related in this course
- The Queue — the chapter this module belongs to.
- The Protocol — the keyspace and the Lua layer the controls run on.
- The native module — the
EMQ.*commands the native twins are. - redis-patterns · Time, delay & priority — the delay/schedule/priority family.
- redis-patterns · Reliable queues — the lease/recover machinery the controls extend.