jonnify·redis /redis-patterns/time-delay-priority

R4 · Time, Delay & Priority

The sorted set
as a clock.

One data structure, read two ways: score a job by a future millisecond and the set is a timer wheel; flatten the score and the mint-ordered id serves as the queue.

Six patterns from the sorted-set family — the delayed queue, schedulers, priority, backoff-and-retry, and leaderboards — each with the trade-off that decides when to reach for it. The grounding is EchoMQ's real time machinery (echo/apps/echo_mq): the emq:{q}:schedule set scored by run-at ms, the emq:{q}:pending set scored 0 and ordered by the branded JOB id, and the promote pump that moves due jobs between them — worked through the codemojex consumer.

§1

Why & when

echo/apps/echo_mq

Time is a constraint the queue itself enforces. Reach for a delayed queue when work must run later, not now — a reminder, a retry after a cooldown, a digest at a fixed hour. Reach for a scheduler when the same work recurs on a cadence. Reach for priority when one class of work must jump ahead of another. Reach for backoff when a failure should be retried, but not immediately and not all at once. Each is a position on one axis: when a job becomes eligible, and in what order the eligible jobs are served.

EchoMQ answers all of these with one schedule set and one server clock. A scheduled job is parked on emq:{q}:schedule at its run-at millisecond — a visibility fence, not a second queue. A pump promotes the due range back to pending on the server's own clock, so the delay is measured on the same clock the reaper and the promote pump read, never the caller's.

Run later

A reminder or a digest that fires at a fixed time — the canonical delayed-queue case.

Recur on a cadence

A periodic reconciliation sweep that registers once and mints a fresh job per run — schedulers.

Order ahead

The oldest pending job served first, by the mint order of its branded id — priority without a packed counter.

Retry, spread out

A transient failure retried after a growing, jittered delay so a cohort does not retry in lockstep — backoff.

The schedule fence · a job parks at its run-at score, the pump promotes the due range to pending
schedule set pending set claim promote: ZRANGEBYSCORE -inf now ZPOPMIN (oldest id)

Every pattern in this chapter is a different answer to one question: when does a job become eligible, and once eligible, in what order is it served.

§2

The patterns

five modules · one workshop

Five teaching modules, then a workshop. Each module is a hub with three dives.

built ready to read soon specified
§3

How to apply

name the time constraint · the mechanism follows

The hard part is not the commands; it is choosing the mechanism. Name the time constraint that matters most for a workload, and the mechanism follows.

Pick the constraint · get the mechanism
readout · Select a constraint to see which mechanism matches and the EchoMQ surface it lands on.

There is no best time mechanism, only the one that matches the constraint you cannot relax.

§4

The workshop

echo/apps/codemojex · EchoMQ

The chapter closes with R4.06: schedule codemojex's notification and digest jobs. A game-result notice is enqueued on a fair lane and delivered under a rate limit; over budget, the worker re-enqueues the same job with EchoMQ.Jobs.enqueue_in/5 after the bucket's reported wait — deferred on the bus, durable, not dropped. A retry on a transient delivery failure rides capped exponential backoff with jitter, and a scheduled job that must survive a crash reaches the durable floor.

Notes on Valkey. The promote pump reads the due head with ZRANGEBYSCORE emq:{q}:schedule -inf <now> LIMIT 0 <n> — a range bounded above by the server clock returns exactly the jobs whose run-at time has passed; the -inf floor means "no lower bound". valkey.io/commands/zrangebyscore

§5

Up next

R5–R8

After the clock comes the durable log — the streams and events later chapters replay and archive.

R5soon

Streams & Events

The durable, replayable log.

R6soon

Flow Control

Staying stable under load.

R7soon

Data Modeling

How data lives in RAM.

R8soon

Production & Ops

Running the tier at scale.

§6

References

sources · related

Sources

  • Valkey — Topicsthe engine the connector is gated against; the sorted-set type the whole chapter stands on
  • Redis — ZADDinsert or update a member's score; the single write the schedule, repeat, and leaderboard sets share
  • Redis — ZRANGEBYSCOREthe range query the promote and reap sweeps use to read the due head of the schedule set
  • Redis — ZPOPMINpop the smallest-score member; the read the claim script uses to serve the oldest pending job

Related