jonnify·redis /redis-patterns/queues/atomic-state-machine/states-as-locations

R3.04.1 · Atomic state machine — dive 1

States as locations.

A job's state is not a status field — it is the emq:{q}: key its id lives in. Moving between states moves the id from one structure to another.

Grounding: echo_mq/keyspace.ex · EchoMQ.Keyspace.queue_key/2, job_key/2 · EchoMQ.Jobs writes the row's state field — pending, active, schedule, dead are sorted sets, the row a hash. Engine: Valkey.

Where does a job live in each state? Select a state to light its emq:{q}: set and place the id there.
pendingmint orderJOB0KH…uC activelease scoreJOB0KH…uC schedulerun-atJOB0KH…uC deadmorgueJOB0KH…uC all sorted sets · the id is a member of exactly one the row hash emq:{q}:job:JOB0KH…uC carries state, attempts, payload
readout · Select a state. Each is its own emq:{q}: sorted set; the job's id is a member of exactly one, and the row's state field follows the membership.
§1

A place, not a flag

atomic-updates.md.txt · the value

A naive queue stores the lifecycle as a column: state = "pending", then state = "active". The job sits in one row and a flag says where it is. Two facts can then disagree — where the job actually sits, and what its flag claims — and a crash between writing the flag and moving the job leaves the pair out of sync, with nothing to reconcile them.

EchoMQ stores the lifecycle as membership. Each state is its own emq:{q}: sorted set, and a job's id is a member of exactly one of them. The job's row — a hash at emq:{q}:job:<id> — still carries a state field, but the field follows the membership; the membership is the truth. Asking "what is pending" is a read of one sorted set, in mint order, with no second index.

The location is the state: a job's id sits in exactly one emq:{q}: set, and the row's state field tracks which one.

§2

The four locations and the row

keyspace.ex · queue_key/2

The keys are built by EchoMQ.Keyspace.queue_key(queue, type), which returns emq:{queue}:<type>, and the row key by job_key(queue, branded), which returns emq:{queue}:job:<branded> after gating the id:

emq:{q}:pending
A sorted set, every member scored 0, so byte order is mint order. The waiting line; claim/3 pops the lowest with ZPOPMIN.
emq:{q}:active
A sorted set scored by the lease deadline on the server clock. A claimed job sits here until completed or its lease expires.
emq:{q}:schedule
A sorted set scored by run-at. A delayed or retried job parks here; the promote pump returns it to pending once due.
emq:{q}:dead
A sorted set, the morgue. A job past its max attempts lands here.
emq:{q}:job:<id>
A hash, the row: state, attempts, payload. The state field tracks the set the id lives in.
One transition — which sets does the id leave and enter? Select a transition; the readout names the move and the row write.
readout · Select a transition to see which emq:{q}: set the id leaves, which it enters, and how the row's state is written.

A transition is a move between sets plus a write to the row's state; the id is in exactly one set at every instant, because each move is one atomic script.

§3

The branded JOB id is the member

echo/apps/codemojex · Codemojex.Guesses

The member that moves is the branded JOB id — a 14-character name, a 3-character namespace plus 11 Base62 over a snowflake (epoch 1704067200000). Because the score in pending is constant, the id's own byte order is the queue order, so a job minted earlier sorts ahead with no second index. The id is gated before any key is built: EchoMQ.Keyspace.job_key/2 raises on an ill-formed id, so a malformed member can never reach a key.

In codemojex, Codemojex.Guesses.submit/3 mints the member — EchoData.BrandedId.generate!("JOB") — and enqueues it on the cm queue's player lane; Codemojex.ScoreWorker claims it, moving the id pending → active for the duration of the scoring, then completing it.

# Codemojex.Guesses.submit/3 (game.ex), trimmed — mint the JOB member, enqueue it on the player lane
job = EchoData.BrandedId.generate!("JOB")
payload = :erlang.term_to_binary({:guess, game, player, guess})
Lanes.enqueue(Bus.conn(), "cm", player, job, payload)   # the id enters emq:{cm}: pending

Notes on ValkeyA sorted-set member is stored once; moving a job between states is a ZREM from one set and a ZADD into another, both inside one script so the member is never in neither set nor both — valkey.io/commands/zadd.

the application · the branded JOB id

The id sorts as it was minted

The branded id is typed, ordered, placed and conformant. The runtime asserts the order property at boot — the same vectors the manuscript cites:

docs/echo/bcs/bcs.2.md · branded-id self-check (boot-asserted)frozen
placement("USR0KHTOWnGLuC")  →  234878118            ← native and pure agree
parse("USR0NgWEfAEJfs")      →  {:ok, "USR", 320636799581945856}
decode("USRzzzzzzzzzzz")     →  :error                 ← an overflow is refused
the pattern

Model the lifecycle as membership, so a transition is a move between structures and "what is in state X" is a read of one structure — never a status flag that can fall out of sync.

its EchoMQ application

Each state is an emq:{q}: sorted set, the row's state follows the membership, and the branded JOB id is the member that moves — gated at job_key/2, mint-ordered in pending.

A job's state is the emq:{q}: set its branded id is a member of; a transition moves the member, and the next dive shows that move as one atomic call.

§4

Recap — the place is the state

next → R3.04.2

A job's state is the emq:{q}: sorted set its id lives in — pending, active, schedule or dead — and the row's state field follows the membership. A transition is a move between sets plus a row write. The next dive runs that move as one EVALSHA, with the row's attempts as the fencing token.

§5

References

sources · related

Sources

  • Redis — Sorted sets — the data type each lifecycle location is, scored to order its members.
  • Redis — ZADD — add a member to a state set with a score: the lease deadline, the run-at, or zero for mint order.
  • Valkey — ZADD — the same move on the engine the connector is gated against.
  • Redis — Keyspace — keys, hash tags, and slots; the basis for the braced emq:{q}: grammar.

Related in this course