jonnify·redis /redis-patterns/time-delay-priority/priority-scores

R4.03 · Time, Delay & Priority — module hub

Priority with composite scores.

A sorted set orders by score, then lexicographically within an equal score. Pack the priority tier and the arrival order into one numeric score and a single sorted set becomes a priority queue — one pop returns the right job.

Grounding: echo/apps/echo_mq/lib/echo_mq/lanes.ex · EchoMQ.Lanes.weight/4 · wclaim/3 — the real bus retired numeric per-job priority by design; precedence lives on the lane, verified verbatim on disk.

The prioritized set — six jobs ordered by composite score. Hover or select a job to read its score and pop position; toggle to "priority only" to see the ties a tie-breaker removes.
scored:jobs — six jobs by composite score (smallest first)
readout · Composite score = priority × 0x100000000 + counter. Hover a slot for its tier, counter, and pop position.

Static order — composite: B, D, F (tier 1 by arrival), then A, E (tier 2), then C (tier 3) · priority-only: the three tier-1 jobs tie and fall back to member order, losing arrival order.

§1

How a sorted set orders: score, then member

lexicographic-sorted-sets.md.txt

A Redis sorted set keeps its members in order by a numeric score. Read it low-to-high and the smallest score is at the head. When two members carry the same score, the set falls back to lexicographic order of the member to break the tie. The ordering is two-level by construction: the score first, the member second.

ZADD scores 100 "alice" 200 "bob" 150 "charlie"
ZRANGE scores 0 -1
# Returns: alice, charlie, bob  — sorted by score

The score is the whole ordering axis. Choose what number goes in it and the set's order becomes whatever that number means.

§2

Composite keys: two fields, one ordering key

lexicographic-sorted-sets.md.txt · Composite keys

The composite-key pattern encodes several fields into one ordering key so a single sorted set answers a hierarchical query. In its lexicographic form the fields go into the member string, read by ZRANGEBYLEX:

ZADD events 0 "user:100:1706648400:login"
ZADD events 0 "user:100:1706648500:click"
ZRANGEBYLEX events [user:100: (user:100:\xff   # all user:100 events, ordered by timestamp

The second encoding puts the high-order field in the score and the low in the member. Members sort by score first, then lexicographically within an equal score, so the score can carry a tier while the member carries the order within it. Bucketing falls out: group by score (a category, a priority tier), order within each bucket. A priority queue is the same composite key, encoded entirely in the score.

ZADD hybrid 1 "a:first" 1 "a:second" 2 "b:first" 2 "b:second"
ZRANGE hybrid 0 -1
# Returns: a:first, a:second, b:first, b:second  — bucket by score, order within

A composite key packs more than one field into one ordering key; pack into the member and read with ZRANGEBYLEX, or pack into the score and bucket by tier with the member ordering within.

§3

The numeric composite score: the priority queue

lexicographic-sorted-sets.md.txt · Numeric rankings

A priority queue packs both fields into the score. The score is priority × 0x100000000 + counter. 0x100000000 is 232, so multiplying the priority by it shifts the tier into the high thirty-two bits; the arrival counter sits in the low thirty-two bits. One IEEE double carries both fields exactly — a double holds fifty-three bits of integer precision, ample for small priorities and counters.

The counter comes from an atomic INCR on a per-queue counter key, taken at the moment the job is added. Every add reads a fresh, monotonically rising counter, so two jobs in the same tier get two different low-bit fields: the earlier add gets the smaller counter. A lower priority number means higher precedence — priority 1 produces a smaller high-bit field than priority 2, so its composite score is smaller, and the set keeps it nearer the head.

ZADD scored:jobs 4294967297 "job-A"   # priority 1, counter 1  → 1·2^32 + 1
ZADD scored:jobs 8589934594 "job-B"   # priority 2, counter 2  → 2·2^32 + 2
ZPOPMIN scored:jobs                  # → job-A: the smaller tier wins

The composite score puts the tier in the high thirty-two bits and the arrival counter in the low; the high bits dominate, so the score orders by tier first and arrival second — in one number.

§4

FIFO within a tier, then ZPOPMIN

lexicographic-sorted-sets.md.txt · INCR / ZPOPMIN

Two jobs of the same priority share the same high thirty-two bits. The tie is broken by the low thirty-two bits — the arrival counter. Because the counter rises by one per add, the first job to arrive in a tier carries the smaller counter and so the smaller composite score: strict FIFO within the tier, with no second structure and no scan. Without the counter, a score of priority alone would tie every job in a tier and the set would fall back to member order, which is not arrival order unless the id is itself time-ordered.

A worker takes the next job with ZPOPMIN — pop the member with the smallest score: the highest-priority tier and, within it, the earliest arrival. One command returns the right job and removes it atomically. ZPOPMIN is the reason a lower priority number wins: the formula makes a higher-precedence job a smaller number, and the pop always returns the smallest first.

The counter breaks the within-tier tie in arrival order, and one ZPOPMIN returns the smallest score — highest tier, earliest arrival — removing it atomically.

§5

When a composite score — when lexicographic instead

lexicographic-sorted-sets.md.txt · When to use

A single composite score earns its place when the ordering is fully numeric and one pop must return the right job: a priority queue, a numeric ranking, any case where the high-order field and the tie-breaker are both numbers. One sorted set then replaces a per-tier structure plus a separate FIFO, and the consume is one ZPOPMIN. Reach for the member-packed lexicographic key instead when the fields are strings and the read is a prefix or range query — autocomplete, hierarchical keys, time-series with string ids — where ZRANGEBYLEX over the member is the natural read.

Pack into the score (numeric)Pack into the member (lexicographic)
Priority queues, numeric rankingsAutocomplete, prefix search
Read with ZPOPMIN / ZRANGEBYSCORERead with ZRANGEBYLEX
Tier in high bits, tie-breaker in low bitsFields delimited in one string, zero-padded

Pack into the score when the order is numeric and the read is a pop; pack into the member when the order is a string and the read is a lexicographic range — the same composite-key idea, two encodings.

§6

Applied in EchoMQ: precedence on the lane, not the score

echo_mq/lanes.ex · weight/4 · wclaim/3

Here the applied turn differs from the textbook, and the difference is the lesson. The real EchoMQ bus carries no numeric per-job priority — it is retired by design. EchoMQ.Lanes states it in the source: "there is no numeric per-job priority (retired by design); ‘served more’ is a property of the identity, not the work," and "‘matters more now’ is a change of lane, mint order is the order theorem."

So EchoMQ does not pack precedence into a job's score. Its pending and lane sets are written at score 0 — ZADD <set> 0 <job-id> — and the branded JOB id (typed, time-ordered) is the whole ordering: byte order is mint order, so a lane serves first-in-first-out by id with no second index. That is the order theorem — the same arrival ordering the composite score reaches with a counter, EchoMQ reaches for free.

Precedence between identities is the per-lane weight. EchoMQ.Lanes.weight/4 writes a gweight hash (group → weight); EchoMQ.Lanes.wclaim/3 runs the weighted rotation: one LMOVE rotates the ring of serviceable lanes exactly once, then serves that lane a fair share of K = min(weight, lane depth, glimit headroom) heads in one atomic turn. A higher-weight lane is served proportionally more over a window, never all of it — fairness is constructed by the rotation, not encoded in a number.

# EchoMQ.Lanes.weight/4 (lanes.ex) — set a lane's fair-share weight, verbatim
def weight(conn, queue, group, w) when is_integer(w) and w >= 1 do
  _ = lane_key!(queue, group)
  keys = [Keyspace.queue_key(queue, "gweight")]
  case Connector.eval(conn, @gweight, keys, [group, Integer.to_string(w)]) do
    {:ok, 1} -> :ok
    other -> other
  end
end

Notes on ValkeyA sorted set orders by score, then lexicographically within an equal score — so EchoMQ's score-0 lane sets order purely by member, and a time-ordered branded id makes that mint order — valkey.io/topics/sorted-sets.

the application · codemojex · a fair lane per chat

The lane, not the message, carries precedence

The consumer is codemojex, the Telegram emoji game on the same stack. Codemojex.Notifier.notify/3 enqueues every notification on EchoMQ.Lanes.enqueue(conn, "cm.notify", chat_id, job_id, payload) — a fair lane keyed by chat id, so one chat's burst cannot starve others — and Codemojex.NotificationWorker drains the lanes by the rotation. Precedence is the chat's lane and its weight, never a score on the message. The lane ring, the fair-share weight, and the order theorem in depth are the dedicated Queue pillar of the EchoMQ course.

echo_mq/lanes.ex · the weighted rotation (@gwclaim), trimmedfrozen
local g = redis.call('LMOVE', KEYS[1], KEYS[1], 'LEFT', 'RIGHT')   ← rotate the ring once
local w = tonumber(redis.call('HGET', ARGV[1] .. 'gweight', g) or '1')  ← the lane's share
local k = w
if depth < k then k = depth end                                   ← never over-pop the lane
-- then ZPOPMIN k heads of THIS lane, on one server-clock lease
§7

The three dives

pack → tie-break → pop

Each dive takes one layer of the composite priority score, then sets it beside the EchoMQ choice: the number that packs both fields, the counter that orders within a tier, and the pop that reads them — with the lane rotation that reaches the same head without a packed score. Read them in order: the packing first, the FIFO second, the pop third.

R4.03.1built

Packing two keys in one score

The core move: priority × 0x100000000 + counter puts the tier in the high thirty-two bits and the arrival counter in the low; changing the tier dominates the counter.
R4.03.2built

FIFO within a tier

Two jobs of the same priority order by the arrival counter; the counter breaks the tie, and EchoMQ reaches the same order for free with a score-0 set and a time-ordered id.
R4.03.3built

ZPOPMIN

Consumption: ZPOPMIN pops the smallest score (highest tier, earliest arrival); why a lower number wins, and the lane rotation that serves the same head by weight.
the pattern

A textbook priority queue packs the tier and the arrival order into one score so a single ZPOPMIN returns the right job — precedence encoded in the number.

its EchoMQ application

EchoMQ moves precedence off the job: arrival is the branded id's mint order (score-0 sets, the order theorem), and tier precedence is the lane's gweight served by wclaim/3 over a rotating ring. The precedence is on the identity, not the number.

Same goal — the right work served first, fairly — reached two ways: pack the precedence into the score, or put it on the lane and let a weighted rotation serve the share.

After R4.03, the Time, Delay & Priority chapter continues with backoff-and-retry and leaderboards (a numeric ranking on the same machinery), closing with the scheduler workshop; fairness and starvation are R6.02 · Priorities and fairness, a later chapter. Return to the Time, Delay & Priority chapter to continue.

§8

References

sources · related

Sources

  • Redis — Sorted sets — score-then-member ordering, the basis of the composite priority score.
  • Redis — ZADD — score a member by its composite priority score; the write that adds a prioritized job.
  • Valkey — ZPOPMIN — pop the smallest score: highest-priority tier, earliest arrival, removed atomically; the engine EchoMQ runs on.
  • Redis — INCR — the atomic arrival counter that breaks the within-tier tie in the composite-score form.
  • Redis — ZRANGEBYLEX — the source's other half: member-packed lexicographic ordering, the contrast to score-packing.
  • Valkey — Sorted sets — the engine's own reference for the score-then-member order EchoMQ's score-0 lane sets depend on.

Related in this course