jonnify·redis /redis-patterns/time-delay-priority/delayed-queue/score-is-fire-time

Redis Patterns/R4/The delayed queue/The score is the fire-time

R4.01.1 · The delayed queue — dive 1

The score is the fire-time.

The score of a delayed job is the time it should run. The textbook puts a raw timestamp in it; EchoMQ scores the schedule set by the run-at millisecond — on the server clock for a relative delay.

Grounding: echo/apps/echo_mq/lib/echo_mq/jobs.ex · enqueue_at/5 · enqueue_in/5 · the @schedule and @promote scripts — the real schedule set, on the server clock.

Schedule, then promote. Select a step to light its path and read the command it runs — a scheduled job parks on the schedule set, then promotes onto the pending set when due.
applicationmints a JOB id schedule setscored by run-at pending setscored 0, mint-ord ZADD schedule ZREM + ZADD
readout · Select a step. A scheduled job parks on the schedule set at its run-at score; the promote sweep moves it onto the pending set when due.

Static verdicts — enqueue_at / enqueue_in → HSET … state scheduled then ZADD schedule <run-at> <JOB id> · promote → ZREM schedule, then ZADD pending 0 <JOB id>, HSET … state pending.

§1

The textbook score: the fire-time itself

delayed-queue.md.txt · the score

The classic delayed-queue pattern puts the timestamp at which the task should run directly into the score. The set sorts by it, so the earliest-due task is at the head, and a range query bounded by the current clock returns the due tasks.

ZADD delayed_queue 1706649000 "task:abc123"   # score = the fire-time, second-for-second

The fire-time of a deferred task is now + delay — to defer five minutes, score by now + 300. One number, one meaning: when this task should run. The one weak spot is the tie. When two jobs share a tick they take the identical score, and a sorted set falls back to lexicographic member order, not arrival order. The question every real scheduler answers is what breaks that tie — and that is what EchoMQ moves on.

score
The run-at time: the moment the job becomes eligible to run.
member
The job identifier; in EchoMQ a branded JOB id.
head
The lowest-scored member: the earliest-due job.

The score carries the meaning. For a delayed job that meaning is its run-at time, and the set orders its members by it — earliest first.

§2

EchoMQ’s score: the run-at millisecond, on the server clock

jobs.ex · @schedule

EchoMQ keeps the structure and prices the score precisely. The schedule set is emq:{<queue>}:schedule (EchoMQ.Keyspace.queue_key(queue, "schedule")), and the score is the run-at millisecond. There are two ways to say when: enqueue_at/5 takes an absolute run-at ms — the caller names the instant, and that integer is the score; and enqueue_in/5 takes a relative delay, with the run-at score computed wire-side from the server’s own clock.

Move the slider below to set a job’s delay against a fixed server now. The readout computes the run-at score for enqueue_in — now + delay — and contrasts it with the absolute enqueue_at score, confirming the relative score is read off the server TIME, not the caller’s.

The run-at score — enqueue_in vs enqueue_at. Set the delay; the readout prices both scores.
5000

Static verdicts — enqueue_in → score = now + delay, now from the server TIME · enqueue_at → score = run_at_ms, the caller’s absolute millisecond.

-- @schedule (jobs.ex) — the relative run-in score, on the server clock (real)
local t = redis.call('TIME')
local now = t[1] * 1000 + math.floor(t[2] / 1000)   # the server clock, in ms
score = now + tonumber(ARGV[4])                      # run-at = now + delay

Notes on ValkeyTIME returns the server’s clock as seconds and microseconds, evaluated inside the script — so a relative delay is priced by the server, never a client whose clock runs fast — valkey.io/commands/time.

The score prices the wait: enqueue_at by the caller’s absolute millisecond, enqueue_in by now + delay read off the server clock inside the script.

§3

One atomic write, and the mint-ordered sort key

jobs.ex · @schedule / @promote

The score is half the story; the other half is that the schedule entry and the job row are written together. A half-written scheduled job — a row with no schedule entry, or a schedule entry pointing at no row — either never runs or runs against nothing. EchoMQ’s @schedule script writes both in one EVAL:

-- @schedule (jobs.ex) — admit a scheduled job atomically (real)
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then          # the id must be JOB-namespaced
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
if redis.call('EXISTS', KEYS[1]) == 1 then return 0 end   # idempotent: a duplicate no-ops
redis.call('HSET', KEYS[1], 'state', 'scheduled', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], score, ARGV[1])          # park the JOB id on the schedule set
return 1

KEYS[1] is the job row, KEYS[2] the schedule set — both declared, both braced emq:{<queue>}: keys that pin one cluster slot. There is a second ordering the score never carries. When a due job is promoted it moves onto the pending set, and that set is scored 0 for every member — its members are the branded JOB ids themselves, so byte order is mint order. The schedule set orders by when a job runs; the pending set orders by when the job was minted.

-- @promote (jobs.ex) — a due job lands on the pending set at score 0 (real)
redis.call('ZADD', KEYS[2], 0, id)                # pending: same score, members are JOB ids
redis.call('HSET', jk, 'state', 'pending')

A branded JOB id is a 14-character name whose text sorts as its mint instant (a snowflake under the JOB namespace), so a job minted earlier but scheduled later sorts, once promoted, by its mint — the queue carries no second index. Use the control below to schedule two jobs, watch them parked in run-at order on the schedule set, then step the clock so the due one promotes onto the pending set.

Schedule, then promote when due. Select a clock to see which jobs are scheduled and which have promoted.

Static verdicts — a job whose run-at is at or below the clock has promoted to the pending set (state pending), where its branded id is the sort key; the rest wait on the schedule set (state scheduled).

The schedule entry and the row are one atomic write, and once promoted a job inherits the mint order of its branded id on the pending set — the score prices the wait, the id prices the queue.

§4

In EchoMQ — the schedule score, in real code

echo_mq/jobs.ex · enqueue_at / enqueue_in

The whole score is real code in echo/apps/echo_mq/lib/echo_mq/jobs.ex. The schedule set is EchoMQ.Keyspace.queue_key(queue, "schedule") → emq:{<queue>}:schedule. enqueue_at/5 parks the id at the caller’s absolute millisecond; enqueue_in/5 computes now + delay from the server TIME inside the @schedule script. Both write the row state = scheduled and ZADD the id onto the schedule set in one atomic EVAL. The id itself, mint-ordered under the JOB namespace, becomes the sort key the moment promotion moves it onto the score-0 pending set.

the bus · the EchoMQ schedule set, in depth

The schedule set and the promote pump

This dive cites the @schedule and @promote bodies as proof the run-at score ships. The full scheduler subsystem — the metronome that beats the promote, the lifecycle controls, the repeatable-job upsert — is the dedicated EchoMQ Queue course. From here, open onto the Queue pillar; this dive teaches the score, that course teaches the scheduler that reads it.

the pattern

A delayed job is scored by its fire-time, so the sorted set orders its members by when they are due.

its EchoMQ application

enqueue_at/5 scores the schedule set by the caller’s run-at ms, enqueue_in/5 by now + delay on the server clock; both write state = scheduled and ZADD emq:{<queue>}:schedule in one atomic script, and the promoted job inherits the mint order of its branded JOB id.

The score prices the wait — the run-at millisecond, read off the server clock for a relative delay — and the branded id prices the queue, so a delayed job is ordered first by when it runs and then by when it was minted.

§5

References

sources · related

Sources

  • Valkey — ZADD — add a member with its score; the single write that parks a scheduled job by its run-at millisecond.
  • Redis — Sorted sets — the data type behind the schedule set: members ordered by a numeric score.
  • Valkey — TIME — the server clock enqueue_in reads inside the script, so a relative delay is priced by the server.
  • Redis — ZRANGEBYSCORE — the range query the same score is later read by, the next dive’s sweep.

Related in this course