jonnify·redis /redis-patterns/time-delay-priority/schedulers/upsert-no-duplicates

R4.02.2 · Schedulers — dive 2

One entry per name: the upsert.

A reboot re-registers the same scheduler. The EXISTS-guarded register keeps exactly one registry entry per registration name — no duplicate recurring job on boot.

Grounding: echo/apps/echo_mq · repeat.ex (EchoMQ.Repeat, @register, verbatim) · the EXISTS probe on the record key, returning {:ok, :exists} for a live name.

Two boots, one registration. Hover or select a boot to light its register call; the second finds the record present and writes nothing.
serviceboots, re-registers emq:{reports}:repeatZSET · 1 entry register → HSET + ZADD register → EXISTS=1, no write
readout · Select a boot. The first register writes one entry; the second finds the record present and writes nothing.

Static verdicts — boot #1 → record absent, HSET + ZADD run, {:ok, :registered}, registry size 1 · boot #2 → EXISTS = 1, the script returns 0, {:ok, :exists}, registry size still 1.

§1

The naive add duplicates

delayed-queue.md.txt · applied to recurrence

A service that owns recurring schedules registers them at startup. Restart the service and it registers them again. The danger is a second registry entry for a registration that already exists: the recurring job would then fire twice.

A naive registration appends. Boot once: the registry has one entry. Boot again without checking, and a register that wrote a fresh record under a recomputed first-run time would corrupt the live cadence. The cure is the idempotent register — probe the record key, and write nothing when it is already present.

register/6
Register a name with a period and a payload template, idempotently.
{:ok, :registered}
The name was fresh; the record and registry entry were written.
{:ok, :exists}
The name was live; nothing changed.

The failure to rule out is a second registration corrupting a live cadence on boot. The EXISTS guard rules it out at the record key.

§2

EXISTS guards, HSET + ZADD store

repeat.ex · @register

The register is one inline Lua script with the guard at the top. First the probe: redis.call('EXISTS', KEYS[2]) returns 1 when the record hash emq:{q}:repeat:<name> already exists, or 0 when the name is new. On 1 the script returns 0 and writes nothing — no record, no registry entry, no change. On 0 it runs the store: HSET the record hash with every_ms and the template, then ZADD the registry with the first-run score against the name.

Because the guard is the first move and both writes are in the same script, two boots of the same name produce exactly one record and exactly one registry entry. The registry's cardinality stays at one entry per registration name across every boot.

The probe is the idempotence: it reads the prior record so the store runs only for a fresh name, and the registry never grows a duplicate on boot.

§3

In EchoMQ — the EXISTS guard in @register

repeat.ex · @register

The real idempotence lives in the @register script of EchoMQ.Repeat. The EXISTS probe runs before any write, so a re-register of a live name returns early and touches nothing:

echo/apps/echo_mq · repeat.ex · @register (verbatim)frozen
if redis.call('EXISTS', KEYS[2]) == 1 then
  return 0                                  -- live name → {:ok, :exists}, no write
end
redis.call('HSET', KEYS[2], 'every_ms', ARGV[2], 'template', ARGV[3])
redis.call('ZADD', KEYS[1], tonumber(ARGV[4]), ARGV[1])
return 1                                    -- fresh name → {:ok, :registered}

A return of 0 is mapped by register/6 to {:ok, :exists}; a return of 1 is {:ok, :registered}. KEYS[2] is the record hash and KEYS[1] is the registry sorted set, both {q}-hashtagged onto the queue's slot. The mirror move is cancel/3: it ZREMs the registry member and DELs the record, so a cancelled name is gone from the declared keyspace and a later register treats it as fresh again.

Notes on ValkeyEXISTS is a single read the script runs before any write, so the register is atomic: the probe and the two writes execute as one script with no interleaving — valkey.io/commands/exists.

/echomq/queue · the scheduling controls in depth

Register racing a sweep

How a register races the pump's sweep, how a cancel mid-tick is reconciled, and how the scheduler-vs-delayed state is kept consistent across a worker pool is the scheduler subsystem the EchoMQ course covers. This dive teaches the upsert; the Queue pillar teaches the controls around it. See /echomq/queue.

the pattern

A scheduler must survive re-registration without re-firing. Probe for a prior entry, and write only when the name is fresh — one entry per registration, across any number of boots.

its EchoMQ application

@register guards on EXISTS KEYS[2]; a live name returns 0 ({:ok, :exists}), a fresh name runs HSET + ZADD ({:ok, :registered}).

A textbook delayed queue adds a fire-time and forgets it. A scheduler keeps exactly one entry per name; the EXISTS guard is what makes the second boot a no-op.

§4

Recap — probe, then store

next → R4.02.3

The register probes the record key with EXISTS and writes only for a fresh name, so the registry holds one entry per registration name no matter how many boots register it. A live name answers {:ok, :exists} and changes nothing. The next dive rewrites that single entry each cycle — start-to-start, anchored to the period.

§5

References

sources · related

Sources

  • Redis — EXISTS — the probe that reads whether the record is present and makes the register idempotent.
  • Redis — ZADD — the registry store; re-adding the same member rewrites, but the guard runs first.
  • Redis — HSET — the record hash carrying every_ms and the payload template per registration.
  • Valkey — Sorted sets — member uniqueness keeps the registry at one entry per registration.

Related in this course