R4.02 · Schedulers
The delayed job that
re-schedules itself.
A repeatable job is a delayed job that re-schedules itself: a sorted set holds one entry per registration, scored by the next run time, and each time an occurrence fires the next timestamp is computed and the same entry's score is rewritten — so the recurrence survives restarts without ever duplicating.
Three dives' worth of clock arithmetic: naming the cadence, the upsert that keeps one entry
per registration, and the start-to-start advance. The grounding is EchoMQ.Repeat
(echo/apps/echo_mq) — the emq:{q}:repeat registry, the EXISTS-guarded
register, and EchoMQ.Pump's per-tick sweep that mints a fresh branded JOB id per
occurrence.
What a scheduler adds over a one-shot delay
echo/apps/echo_mq · repeat.exA one-shot delayed job is a member in a sorted set scored by its fire-time. It runs once and is gone. A scheduler carries that one move forward: it keeps a registry of recurring schedules, one entry per registration, each scored by when it next runs. When an occurrence fires, the next run time is computed and the same entry's score is rewritten. The score is still a future timestamp, exactly as in the delayed queue — the new idea is that the entry is rewritten each cycle instead of consumed once.
A cadence
A rule for "when next" — a wall-clock cron slot or a fixed interval (every_ms), evaluated to a millisecond.
A registry
A separate sorted set, scored by the next run time, holding one entry per recurring registration.
An upsert
Registering a name that already exists leaves its entry alone rather than appending a duplicate.
A fresh job per fire
Each occurrence mints a new branded JOB id — a first-class, browsable, mint-ordered job, never a reused row.
Every dive in this module is one answer to the same question: a single registry entry, scored by the next run — how is that score named, kept unique, and rewritten.
The repeat registry & its upsert
repeat.ex · @registerNaming the cadence. There are two ways to name when next. A cron expression
(0 9 * * *) names a wall-clock calendar position; a fixed interval (every_ms: 900000)
names a gap. Both reduce to a single millisecond timestamp, which is the score the registry stores.
EchoMQ.Repeat is the interval form: it carries an every_ms period and advances the
score by it each cycle.
The registry. EchoMQ.Repeat declares two {q}-hashtagged keys, so a
queue's repeat state lands on its own cluster slot: emq:{q}:repeat — a sorted set scored by the
next-run millisecond, members the registration names — and emq:{q}:repeat:<name> — a hash
carrying every_ms and the payload template. The registry carries only the next-run
times; the period and template live in the companion hash.
The upsert (no duplicate on boot). A service that boots re-registers all its schedules. A
naive add would push a second entry and the recurring job would fire twice. The @register script
probes the record key first — if redis.call('EXISTS', KEYS[2]) == 1 then return 0. A present
record returns 0 and writes nothing; register/6 answers
{:ok, :exists}. Only a fresh name runs the HSET and the ZADD, answering
{:ok, :registered}. One name, one record, one registry entry, across any number of boots.
Producing the next run (start-to-start). The registry holds only the next run. The cadence
is EchoMQ.Pump's: each tick it reads the due registrations
(ZRANGEBYSCORE emq:{q}:repeat -inf <now>), mints a fresh JOB id and enqueues
each occurrence, then advances the score. advance/4 sets the new score to
now_ms() + every_ms — start-to-start, measured from the release instant plus the period, not from
a finish time. A slow occurrence does not push the cadence later.
Notes on Valkey. The pump's due read is ZRANGEBYSCORE emq:{q}:repeat -inf <now>
LIMIT 0 <n> — the names scored at or before now, oldest-due first, bounded per tick. The score is the
sort key, so "due" is a range query, not a scan. valkey.io/commands/zrangebyscore
When to use, when to avoid
the position on the trade-offUse — a calendar or fixed cadence
Periodic reports, cleanup sweeps, polling an upstream — recurring work named once.
Use — restart-safe recurrence
Schedules that must survive a restart without re-firing or duplicating; the registry is on the bus, not in a process.
Avoid — sub-second cadences
A sorted-set registry swept by a pump on a beat is millisecond-grained, not a real-time scheduler.
Avoid — millions of one-offs
Enormous fan-outs of distinct future runs are delayed jobs, not registrations — they belong in the schedule set.
Full calendar correctness across timezones and daylight-saving transitions is a parsing depth that belongs to EchoMQ's scheduler subsystem — the EchoMQ course's territory, not this module's.
The three dives
name · keep unique · rewriteThree dives, one per move on the single registry entry: how the next-run score is named, how it is kept unique across boots, and how it is rewritten each cycle.
Cron vs interval
Two ways to name "when next"; both become one timestamp, and EchoMQ.Repeat is the interval form.
The upsert, no duplicates
A reboot re-registers the same name; the EXISTS guard keeps one entry per registration.
Start-to-start cadence
advance/4 rewrites the score to now + every_ms — anchored to the period, not the finish.
In EchoMQ — the registration that does not duplicate
repeat.ex · @registerA textbook delayed queue can already schedule one future run. A scheduler makes the run re-schedule itself.
EchoMQ.Repeat keeps exactly one emq:{q}:repeat entry per registration name and makes
the register idempotent with the EXISTS probe, so a restart re-registering the same name finds
the record already present and changes nothing rather than spawning a duplicate. Registration is one inline
Lua script; the guard is its first move, and only a fresh name runs the two writes:
if redis.call('EXISTS', KEYS[2]) == 1 then return 0 -- already registered → {:ok, :exists} end redis.call('HSET', KEYS[2], 'every_ms', ARGV[2], 'template', ARGV[3]) redis.call('ZADD', KEYS[1], tonumber(ARGV[4]), ARGV[1]) return 1 -- fresh name → {:ok, :registered}
KEYS[1] is emq:{q}:repeat (the registry sorted set), KEYS[2] is
emq:{q}:repeat:<name> (the record hash), both gated onto the queue's slot. The score is the
next-run millisecond; the member is the registration name. EchoMQ.Pump then sweeps the set each
tick, minting a fresh branded JOB id per occurrence and advancing the score by
every_ms start-to-start. The full cron parser, the timezone handling, and the
scheduler-vs-delayed reconciliation across a worker pool belong to the EchoMQ course's Queue pillar.
The Queue, in depth
The emq:{q}: queue protocol that the repeat registry rides — claim, retry, and the scheduling controls that maintain the registry, taught in depth in the Queue pillar.
When a fired job is archived
A scheduled occurrence that is archived for replay reaches the durable floor — ETS head over Valkey over the page engine, the durability dial the persistence course walks down.
References
sources · relatedSources
- Redis — ZADDwrite the next-run millisecond as the registry score; re-adding the same member rewrites it
- Valkey — ZRANGEBYSCOREthe pump's due read: the names scored at or before now, bounded per tick
- Valkey — Sorted setsthe registry as a sorted set of next-run times, on the engine the connector is gated against
- Redis — EXISTSthe idempotence probe: a present record makes the register a no-op
Related
- R4.02.1 · Cron vs intervalnaming the cadence
- R4.02.2 · The upsertone entry per registration name
- R4.02.3 · Start-to-startrewriting the single next-run score
- The Queue, in depththe EchoMQ protocol behind the registry