jonnify·redis /redis-patterns/queues/processing-list

R3.01 · Reliable Queues — module hub

Processing list.

Guarantee at-least-once message delivery by atomically transferring a job from the waiting queue into a processing list, so recovery is possible if a consumer crashes before completing the work.

A naive queue pops a job off with RPOP and is one crash away from losing it: between the pop and the finish the job exists nowhere. This module is the move that prevents it — take the job and record it as in flight in one indivisible step. The grounding is the real EchoMQ worker path (echo/apps/echo_mq): the leased claim, EchoMQ.Jobs.claim/3 over the emq:{queue}:pending and emq:{queue}:active sorted sets. The three dives take the move apart — the list as a queue, the atomic transfer command, the in-flight list as the recovery ledger.

The in-flight move — remove loses the job, move keeps it recoverable. Select how the worker takes job 7, then read where a crash leaves it.
pending setemq:{q}:pending worker— active setemq:{q}:active ZPOPMIN → ZADD RPOP, held in worker job 7 — select a mode
readout · Select a mode. The move parks job 7 in the active set (recoverable); a plain RPOP removes it before any work runs (lost on a crash).
§1

The problem with simple queues

reliable-queue.md.txt · the gap

A naive queue runs four steps. A producer pushes a job with LPUSH queue "message". A consumer takes it with RPOP queue, which removes it from Redis. The consumer then crashes during processing. The job is gone permanently — not in the queue, not finished, with nothing to recover it from.

The flaw is the RPOP. It removes the job before any work has run, so between the pop and the finish there is a window in which the job is in neither place. A deploy, an out-of-memory kill, or a power loss in that window drops the job. The window is small, but a busy queue runs the move millions of times, and small windows fire.

A simple queue loses a job on a crash because RPOP removes it before the work is done; the cure is to move the job, not remove it.

§2

The solution: atomic transfer

reliable-queue.md.txt · move, do not remove

Instead of removing the job, atomically move it into a "processing" list. The job stays in Redis until it is explicitly removed after successful processing. The transfer is one command — RPOPLPUSH, or the modern LMOVE — so there is no instant at which the job is in neither place.

A crashed worker therefore leaves the job parked in the processing list, where it can be found and returned. The naive form removes the job and bets the worker survives; the atomic form moves the job and keeps it recoverable whether the worker survives or not. That single change — move, do not remove — is the whole pattern.

Move the job into a processing list rather than removing it from the queue; it stays in Redis, recoverable, until the work is acknowledged.

§3

How it works

reliable-queue.md.txt · the move + the ack

The consumer uses LMOVE (or the older RPOPLPUSH) to atomically transfer a job from the main queue to a processing queue. The one command removes the job from the right of work_queue, adds it to the left of processing:worker1, and does both atomically — no job loss is possible.

LMOVE work_queue processing:worker1 RIGHT LEFT   # atomic dequeue into the processing list
# RPOPLPUSH src dst  ==  LMOVE src dst RIGHT LEFT

The older RPOPLPUSH source dest is the same move in fixed directions. After successful processing, remove the job from the processing queue with LREM processing:worker1 1 "message". For consumption without polling, the blocking form BLMOVE work_queue processing:worker1 RIGHT LEFT 30 waits up to 30 seconds for a job rather than returning immediately on an empty queue. The second dive walks RPOPLPUSH vs LMOVE and the blocking variant.

LREM processing:worker1 1 "message"   # the ack, after the work is truly done

Notes on ValkeyZPOPMIN pops the lowest-scored member of a sorted set and returns it atomically — the take inside EchoMQ's leased claim, the sorted-set form of popping the oldest job — valkey.io/commands/zpopmin.

One command — LMOVE or RPOPLPUSH — does the pop and the push atomically; the ack is a separate LREM only after the work is done.

§4

The processing lifecycle

reliable-queue.md.txt · dequeue · process · ack

A job moves through three steps. Dequeue: LMOVE work_queue processing:worker1 RIGHT LEFT. Process: the application handles the job. Acknowledge: LREM processing:worker1 1 "message". If the worker crashes between the dequeue and the ack, the job remains in processing:worker1. That parked job is the seam reliability rests on: still in Redis, attributable to a worker, returnable for another attempt. Delivery is therefore at-least-once — delivered one or more times, never zero — which R3.02 makes the consumer idempotent for, and R3.03 reclaims a stalled job with. This module stops at the move.

StepCommand
DequeueLMOVE work_queue processing:worker1 RIGHT LEFT
Processthe application handles the job
AcknowledgeLREM processing:worker1 1 "message"

Dequeue, process, acknowledge — a crash between the dequeue and the ack leaves the job in the processing list, recoverable, which is what makes delivery at-least-once.

§5

Per-worker processing queues

reliable-queue.md.txt · the recovery ledger

Each worker should have its own processing queue — processing:worker1, processing:worker2. A per-worker list lets a recovery monitor attribute a parked job to the worker that holds it, and return it cleanly. The processing list is not only a buffer; it is the recovery ledger, a list of exactly the jobs currently in flight per worker. EchoMQ records the lease deadline as the in-flight set's score instead, so the score is the recovery clock and no per-worker list is needed — the third dive reads that ledger.

Give the in-flight place a way to name when each job is due back — a per-worker list, or a lease score — so a recovery monitor can return a parked job; the in-flight place is the recovery ledger.

§6

The three dives

primitive → command → ledger

Each dive takes one layer of the move, in source order: the list as a queue underneath everything, the atomic transfer command that closes the gap, and the in-flight list read as the recovery ledger. Read them in order — the primitive first, the command second, the ledger third.

the pattern

Move a job into a processing list in one atomic step, so it is in exactly one place at every instant and a crashed worker leaves it parked, recoverable, never lost.

its EchoMQ application

EchoMQ.Jobs.claim/3 runs one inline @claim script — ZPOPMIN emq:{queue}:pending, HINCRBY the attempts token, HSET state active, ZADD emq:{queue}:active at the lease deadline — the pop-and-record move with no gap, the real worker path.

Redis offers one move that never loses a job; EchoMQ's queue chose it — the leased claim — and proves the pattern is real.

/echomq/queue

A door, not a depth — the EchoMQ Queue pillar

This module cites one excerpt of EchoMQ's protocol — the leased @claim script inside EchoMQ.Jobs.claim/3 — as proof the move ships. The full worker fetch loop, the eight inline scripts of the leased state machine, the attempts fencing token, and the consumer's parked BLPOP wake are the subject of the dedicated EchoMQ course. From here, open onto the Queue pillar. This module teaches the move; that course teaches the engine that runs it.

After R3.01, the Reliable Queues chapter continues with at-least-once delivery and an idempotent consumer, then stalled recovery, closing with the EchoMQ worker-path workshop. Return to the Reliable Queues chapter to continue.

§7

References

sources · related

Sources

Related in this course