R2.01.1 · Atomic updates — dive 1

Optimistic locking with WATCH.

Watch the key, read it, queue the write, and commit only if it has not changed. A conflict aborts the transaction; the loser retries.

Grounding: the generic optimistic-concurrency primitive, contrasted with EchoMQ’s single inline Lua EVAL — echo_mq/lib/echo_mq/jobs.ex · echo_wire/connector.ex eval/5.

The CAS retry loop — withdraw 30 from 100. Select an attempt to light its timeline and read whether EXEC committed.

readout · Select an attempt. Attempt 1 aborts when a write lands inside the window; attempt 2 reads the new value and commits.

Static verdicts — attempt 1 → a SET 60 lands between WATCH and EXEC, EXEC returns nil, nothing applied · attempt 2 → fresh read of 60, no interleave, EXEC commits, balance 30.

§1

Watch, read, queue, exec

atomic-updates.md.txt · Pattern 1

A bare MULTI/EXEC runs its queued commands with nothing interleaved, but its writes are fixed before the transaction opens. For a read-modify-write that is unsafe: a balance read outside the transaction can be stale by the time the new balance is written. WATCH closes the gap. It marks one or more keys, and the following EXEC commits only if none of the watched keys changed since the WATCH.

The sequence is check-and-set: read the value, compute the next value, and commit only if the value the computation used is still the value in Redis. A change between the WATCH and the EXEC makes EXEC return nil — a discard, not an error. No partial state is left, and the next step is the caller’s to choose.

WATCH account:123:balance              # mark the key
balance = GET account:123:balance      # read the current value
new_balance = balance - 100            # compute outside the transaction
MULTI                                  # open the transaction
  SET account:123:balance new_balance  # queue the write
EXEC                                   # commit — or nil if the key changed

WATCH key…: Mark keys for the next transaction. If any is modified before EXEC, the transaction is discarded.
MULTI: Open the transaction. The commands after it are queued, not run yet.
EXEC: Commit the queued commands — or return nil when a watched key changed since WATCH.
UNWATCH: Drop all watches on the connection without running a transaction. Used when a guard rejects the operation before MULTI.

A discarded transaction is a conflict, not a failure. The fix is to read again and retry — the next section bounds that loop.

§2

The retry loop, bounded

atomic-updates.md.txt · Retry Loop

A nil from EXEC is a signal to try again with a fresh read. Wrap the whole attempt — WATCH, the read, the guard, MULTI, EXEC — in a loop with a retry ceiling. On a nil, loop; on a non-nil result, the write committed and the loop returns. The ceiling turns a pathological live-lock under a hot key into a bounded, surfaced conflict.

max_retries = 5
for attempt in range(max_retries):
    WATCH account:123:balance
    balance = GET account:123:balance
    if balance < amount:
        UNWATCH                        # release the watch; no transaction
        raise InsufficientFunds()
    new_balance = balance - amount
    MULTI
      SET account:123:balance new_balance
    result = EXEC
    if result is not None:
        return                         # committed
    # EXEC returned nil — a watched key changed; retry
raise TooManyRetries()

The guard reads inside the watched window and calls UNWATCH before raising, so a rejected withdrawal leaves no watch dangling on the connection. The ceiling bounds the cost: each retry is one more read and one more round trip, so an unbounded loop under a hot key would spin. A bounded loop converts that into a conflict the caller can report.

The hero above runs this loop over a fixed schedule. On attempt 1 another client writes the balance inside the watched window, so EXEC returns nil and the attempt is discarded. Attempt 2 reads the new value and finds a clean window, so EXEC commits.

A discard is not an error. The retry loop reads again and re-attempts, and a ceiling makes a hot key a surfaced conflict instead of an infinite spin.

§3

Watching more than one key

atomic-updates.md.txt · Multi-Key WATCH

WATCH takes several keys, and EXEC discards if any one of them changed. That makes a multi-key read-modify-write atomic: confirm stock and the order status together, then decrement stock and confirm the order in one transaction — or commit nothing.

WATCH inventory:sku123 order:456:status
stock = GET inventory:sku123
if stock < quantity:
    UNWATCH
    raise OutOfStock()
MULTI
  DECRBY inventory:sku123 quantity
  SET order:456:status "confirmed"
EXEC

Multi-key WATCH — confirm an order against a stock of 8. Select an outcome and set the order quantity; the readout names what committed.

order quantity 3

readout · Select an outcome. With no concurrent change the order confirms and stock drops; a concurrent purchase aborts the EXEC.

Static verdicts — no concurrent change → both writes apply, order confirmed, stock decremented · concurrent purchase → a SET inventory lands inside the window, EXEC returns nil, neither write applied, retry.

Multi-key WATCH carries one cluster constraint: in a Redis Cluster every watched key must live on the same hash slot, because a transaction runs on a single node. The keys are co-located with a hash tag — inventory:{sku123} and order:{sku123}:456 share the {sku123} slot. Colocation is its own module later in this chapter; here the rule is the boundary: a multi-key WATCH transaction is a single-slot operation.

Notes on ValkeyA WATCH/MULTI/EXEC transaction runs on one node, so every watched key must hash to one slot; a hash tag forces colocation — valkey.io/topics/cluster-spec.

One WATCH over several keys makes the whole multi-key write all-or-nothing: any one of them changing aborts the lot.

§4

Optimistic, never held

echo_mq/jobs.ex · connector.ex:63

WATCH is optimistic. It is built for the rare conflict: take no lock, and pay only when a conflict actually happens — with a retry. A pessimistic lock is the opposite trade: acquire a lock, do the work, release it, and serialize every caller whether or not a conflict would have occurred. Optimistic concurrency wins when contention is low and the critical section is short — the common case for a single-key or single-slot read-modify-write. Under a hot key the retries pile up, which is the signal to reach for a Lua script, the next dive, where the whole read-modify-write runs on the server with no retry loop.

EchoMQ takes the script answer for every transition. It does not run a watched transaction with a retry loop. EchoMQ.Jobs.enqueue/4 runs the inline @enqueue script through EchoMQ.Connector.eval/5, and the script holds the server for its whole duration, so no other client interleaves and there is nothing to retry. The whole read-branch-write is one round trip.

the pattern

WATCH/MULTI/EXEC is the read-modify-write where the logic lives in the client: watch the key, compute, and commit only if it has not changed. A conflict is a retry, never a held lock.

its EchoMQ application

The transition path takes the other atomic-RMW tool. enqueue/4 runs the inline @enqueue script — one EVAL via EchoMQ.Connector.eval/5, no watch and no retry loop, because the script itself admits no interleaving.

WATCH trades a retry for never holding a lock: cheap when conflicts are rare, and a script when they are not.

the door · the EchoMQ course

The single-script transition is the next course

This dive cites one contrast — EchoMQ’s single-script transition — to place WATCH among the atomic-RMW tools. The full transition bundle and the EVALSHA dispatch are the subject of the dedicated EchoMQ course. The script tool itself is the next dive, R2.01.2 · Lua for logic.

§5

Recap — watch, commit, or retry

next → R2.01.2

Optimistic locking with WATCH is one read-modify-write tool: mark the key, read it, queue the write in MULTI, and EXEC commits only if nothing touched the watched keys. A change aborts the transaction and the bounded retry loop re-attempts with a fresh read. Several keys can be watched at once for a multi-key write, with the single-slot rule in a cluster. The next dive moves the logic onto the server — a Lua script that runs atomically with no retry at all.

§6

References

sources · related

Sources

Redis — Transactions — MULTI/EXEC, and how WATCH adds optimistic locking to a transaction.
Valkey — WATCH — mark keys so EXEC discards if any of them is modified first.
Redis — MULTI — open a transaction; following commands are queued, not run.
Redis — EXEC — commit the queued commands, or return nil when a watched key changed.

Related in this course

R2.01 · Atomic updatesthe module hub
R2.01.2 · Lua for logicthe script that holds no retry loop
R2 · Coordinationthe chapter
/echomq/protocolthe EchoMQ protocol the single-script transition belongs to
/elixirthe functional-Elixir & OTP craft behind the echo umbrella