R2.04.3 · Cross-shard consistency — dive 3
Make it invisible.
A detector catches the tear after the read. A commit marker keeps the half-finished write from ever being read — the data is present, but switched off until the marker turns it on.
Grounding: echo/apps/echo_mq · EchoMQ.Jobs.complete → @complete (jobs.ex:139) — one Lua EVAL over co-located keys, visible-or-not in one step, so the queue needs no marker. The marker is the multi-shard substitute for that one-EVAL atomicity.
Static verdicts — crash before marker → A and B present, marker absent,
EXISTS = 0, gate closed, data invisible · crash after marker → all three present,
EXISTS = 1, gate open, read A and B safely.
Write the marker last
cross-shard-consistency.md.txt · Commit MarkerThe shared token and the version field both detect a tear: the read pulls both values back, compares them, and reacts to a mismatch. The torn pair still reaches the caller, and the caller has to repair it. A commit marker changes the order of operations so the incomplete pair never reaches a read at all. Write every data key first. Then, as a separate final step, write one extra key that records the decision to commit:
# Phase 1: write the data (may partially fail) SET txn:abc:A "{payload_A}" SET txn:abc:B "{payload_B}" # Phase 2: mark the transaction committed SET txn:abc:committed "1" EX 3600
The ordering is the whole mechanism. Phase 1 places the data; phase 2 publishes it. Until the marker key
exists, the data keys are present but uncommitted. A reader does not touch the data keys until it has confirmed
the marker: EXISTS txn:abc:committed is the gate — a 1 means phase 2 completed,
a 0 means the transaction is incomplete and the data is treated as absent:
if EXISTS txn:abc:committed:
a = GET txn:abc:A
b = GET txn:abc:B # safe to use — the marker proves both writes landed
else:
# transaction incomplete — ignore the data, or wait and re-check
If the writer dies after phase 1 but before phase 2, the data keys exist — possibly torn, A new and B
old — but the marker does not. The gate returns 0, the read never touches the half-written
pair, and the incomplete transaction is invisible. A crash after phase 2 is a complete transaction by
definition: every data key and the marker are present, so the gate returns 1. The
EX 3600 bounds the marker's lifetime so abandoned records do not accumulate without bound.
A commit marker trades an extra key and a round trip for a clean rule: a read returns a committed transaction or it returns nothing, never a half-finished one.
A two-phase-commit shape
cross-shard-consistency.md.txt · the marker as decision recordThe pattern resembles two-phase commit, with the marker acting as the decision record. Phase 1 is the prepare: stage every value, commit to nothing. Phase 2 is the single write that flips the whole transaction from prepared to committed. It is a shape, not a full protocol — there is no coordinator forcing all participants to commit, no rollback that removes the staged data, and the marker write is itself a single key that can fail. What it buys is a clean visibility rule across keys that cannot share a slot.
Stacks with a version. A bare marker answers one question — is this transaction whole? A version field (the previous dive) answers a second — which write is newer? The two are independent: gate the read on the marker, then order the survivors by version.
Marker versus a bare token
cross-shard-consistency.md.txt · ComparisonThe same partial-failure timeline reads two ways. A bare shared token from the first dive is a
detector: the read fetches both values, parses both tokens, and on a mismatch returns a torn pair the
caller must repair. A commit marker is a gate: the read checks EXISTS …:committed first
and, finding it absent, returns nothing. Detection exposes the tear and pushes the recovery onto the reader;
the marker hides the tear and removes the recovery step entirely. The comparison table in the source places
them side by side — a transaction stamp buys detect only, a commit marker buys detect plus
visibility. Run the one timeline through both:
Static verdicts — torn → the detector returns the torn pair (exposed), the marker returns nothing (hidden) · whole → both readers return the whole pair; the difference only shows on a partial failure.
A token detector returns the torn pair and asks the caller to repair it; a commit marker returns nothing and the incomplete write is never seen. Detection exposes the tear; the marker hides it.
Applied — one EVAL removes the marker
echo_mq/jobs.ex:139,313 · connector.ex:63EchoMQ does not write a commit marker on its own queue transitions, because the transition it would gate is
already atomic in one step. A queue's keys share one slot (the {q} hashtag in
EchoMQ.Keyspace.queue_key/2), so the whole move is one inline Lua script.
EchoMQ.Jobs.complete/4 (jobs.ex:313) hands Connector.eval/5 the
@complete script (jobs.ex:139) over two declared keys — KEYS[1] the active set,
KEYS[2] the job row:
local att = redis.call('HGET', KEYS[2], 'attempts') ← emq:{orders}:job:<id>
if not att then return 0 end
if att ~= ARGV[2] then
return redis.error_reply('EMQSTALE complete token mismatch')
end
local was_active = redis.call('ZREM', KEYS[1], ARGV[1]) ← emq:{orders}:active
redis.call('DEL', KEYS[2])
return 1
That whole sequence — the fencing check, the ZREM off the active set, the
DEL of the row — is one EVAL. It is visible-or-not in a single step: the job is
retired in full, or EMQSTALE is returned and nothing changed. There is no half-written pair to
gate, so there is nothing for a commit marker to do. The v2 law that makes this one step safe is in
Connector.eval/5 (connector.ex:63): every key the script touches is declared in
KEYS, so the engine keeps them on one slot and runs the move as a unit.
Notes on ValkeyA Lua script runs atomically: its effects land together or not at all, so a multi-key transition over co-located keys is visible-or-not in one step, with no marker needed — valkey.io/topics/programmability.
One slot, one EVAL — the next module, in depth
The colocation that removes the need for any of these — the marker, the token, the version — is
the chapter's next module, R2.05 hash-tag colocation: the
{tag} hash, CRC16 mod 16384, and the CROSSSLOT prevention. The full v2 script bundle
is taught in depth in the dedicated EchoMQ course. A commit marker is the multi-shard
substitute for keys that genuinely cannot share a slot — a Valkey read-model paired with
codemojex's database of record.
A commit marker is a read-visibility gate: write the data, then the marker, and read only what
EXISTS …:committed confirms, so a half-finished transaction is never visible.
The @complete script on one co-located slot makes a transition visible-or-not in one
EVAL, so no marker is needed. The marker is the multi-shard substitute for that one-EVAL
atomicity — for keys that cannot share a slot.
A commit marker trades an extra key for never serving a half-finished transaction. EchoMQ gets
the same property for nothing by keeping the move on one slot, in one EVAL; the marker is what
stands in when one slot is not an option.
Recap — hide the incomplete
back → R2.04A commit marker reorders the writes so the incomplete pair is never read: write the data, then one final
marker, and read only what EXISTS confirms. It hides the tear where a token or a version exposes
it, at the cost of one extra key and a round trip. EchoMQ needs none of it: the single Lua EVAL
over co-located keys is the serialization point, visible-or-not in one step. Return to the
module hub for the prevention comparison,
or to the Coordination chapter for R2.05.
References
sources · relatedSources
- Redis — EXISTS — the gate the reader checks;
EXISTS txn:id:committedreturns1for a committed transaction and0for an incomplete one. - Valkey — Programmability — atomic Lua execution: a multi-key transition over co-located keys is visible-or-not in one step.
- Redis — SET — the data writes and the marker write;
SET txn:id:committed 1 EX 3600. - Redis — Transactions — the two-phase-commit framing the marker mirrors across keys that cannot share a slot.
Related in this course
- R2.04 · Cross-shard consistencythe module hub
- R2.04.2 · Version tokensthe version a marker composes with
- R2.01 · Atomic updatesthe single-slot atomic move
- R2 · Coordinationthe chapter
- /echomq/protocolthe EchoMQ protocol in depth
- /elixir · CQRSthe single-writer engine