The record hash · dive 03 · the cargo
The payload
The third field is the caller's. The protocol stores it, moves it, delivers it — and never parses it.
The third field, payload, is the caller's data, and no transition parses
it. Admission takes it as ARGV[2] and stores it verbatim; the claim hands it back
unchanged — on either executor. Between those two points the protocol does nothing with it: it is
opaque cargo, addressed, moved, and delivered but never parsed, validated, or branched on. That
is what lets one protocol carry any job for any consumer. Round-trip a sample payload on the right and read it
back, byte for byte.
Enqueue then claim · pick a payload
Opaque cargo
The payload is the caller's, and the protocol holds no opinion about its bytes. It is not parsed on the way in, not validated, not indexed, not branched on; it is stored as one string and returned as the same string. The system's job is to address it, move it through the lifecycle, and deliver it to a worker — not to understand it.
That is deliberate, and it is what makes the protocol general. Because no transition reads into the value, the same queue serves any job for any consumer, in whatever encoding the caller and the worker agree on — JSON, a bare string, a packed binary. The cargo law: the bus carries; the caller decides what.
Both executors keep the law. EMQ.ENQUEUE writes the argument as given in the same
HSET, and EMQ.CLAIM returns the stored bytes as a bulk reply:
the native commands treat the payload as bytes from end to end. update_data/4 can
replace the field whole, but the contents stay the caller's.
The payload is opaque cargo: stored, moved, delivered, never parsed, on either executor. The protocol holds no opinion about the bytes, so it carries any job.
Store and return — two beats
Beat one — the named handle. EchoMQ.Jobs.enqueue/4 requires the
payload to be a binary (when … and is_binary(payload)) and passes it as
ARGV[2]. The guard makes the field always hold a string; the contents stay the
caller's.
# echo_mq — EchoMQ.Jobs: the named handle for admission. # The is_binary(payload) guard means the field always holds a string, but its # CONTENTS are the caller's. The payload rides ARGV[2]; no script parses it. def enqueue(conn, queue, job_id, payload, opts \\ []) when is_binary(job_id) and is_binary(payload) do Keyspace.valid_queue!(queue) via = Keyword.get(opts, :via, Connector) keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")] case via.eval(conn, @enqueue, keys, [job_id, payload]) do {:ok, 1} -> {:ok, :enqueued} {:ok, 0} -> {:ok, :duplicate} {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind} other -> other end end
Beat two — the two scripts. @enqueue stores the payload with its
one HSET, untouched; @claim2 returns it with
HGET, untouched, after deriving the row key from the declared base
(KEYS[3] .. 'job:' .. id). Side by side, the round trip is store-then-return; neither
script parses the value.
-- @enqueue, the store: its one HSET. The payload is ARGV[2], written -- verbatim as the third field. -- … redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2]) -- … -- @claim2, the return: the whole body. The row key derives from the -- declared base KEYS[3]; the payload is read back, never parsed. local popped = redis.call('ZPOPMIN', KEYS[1]) if #popped == 0 then return {} end local id = popped[1] local jk = KEYS[3] .. 'job:' .. id if redis.call('EXISTS', jk) == 0 then return {} end local att = redis.call('HINCRBY', jk, 'attempts', 1) redis.call('HSET', jk, 'state', 'active') local t = redis.call('TIME') local now = t[1] * 1000 + math.floor(t[2] / 1000) redis.call('ZADD', KEYS[2], now + tonumber(ARGV[1]), id) -- The worker is handed HGET payload: byte for byte what admission stored. return {id, redis.call('HGET', jk, 'payload'), att}
Nothing between the HSET and the HGET parses the value. The
protocol's contract over the record is the field names and the meanings of state and
attempts (and, natively, failures and
lease_epoch); payload is a passthrough.
A row with no payload. Every admission writes the field — enqueue, scheduled admission,
lanes and flows all put payload in the same HSET — so a
row without one lies outside the admission path. If a claim meets such a row, both executors reply
[id, nil, att]: EMQ.CLAIM returns a nil for the missing field,
and in the script HGET yields Lua false, which converts to a nil
reply without ending the table. The parity harness leaves this case out of its comparison.
Opaque vs protocol — which fields the protocol interprets
Some fields carry meaning the protocol defines; one it only carries. state and
attempts are protocol fields on both executors — the transitions write
state with the lifecycle positions the protocol defines, and every holder's call
compares attempts with its token. Natively, failures and
lease_epoch are protocol fields too: EMQ.FAIL caps on
failures, and EMQ.STALLED.CHECK compares
lease_epoch with the server's run_id.
payload is the opaque field: a transition moves it and never parses it. Pick a field and
read which the system interprets.
Does the protocol interpret this field?
Keep the payload opaque to the store: let the value be the caller's, and the key and the control fields be the system's. Data Modeling & Memory (R7) teaches the move in Redis Patterns Applied; Patterns become protocol is the near side.
payload is ARGV[2], stored verbatim by
@enqueue and EMQ.ENQUEUE and returned verbatim by
@claim2 and EMQ.CLAIM; the transitions interpret only the
control fields. The cargo law, in both executors.
The protocol interprets state and attempts, and
natively failures and lease_epoch;
payload it only carries. The value is the caller's, so one protocol fits any job.
Recap
payload is the caller's opaque cargo — stored verbatim at admission, returned
verbatim by the claim on either executor, never parsed. The protocol interprets the key and the control fields;
the value is the caller's. That is what lets one protocol carry any job. With all three birth fields read, the
record is complete — the chapter's remaining modules hold this shape fixed with the Lua layer, the native
module and the immutability discipline.
Back to The record hash hub, or on to the rest of The Protocol — the Lua layer, the native module and the immutability that hold the field set fixed.
References
Where the writes that store and return the payload are documented.
Sources
- Valkey — HGET — how the claim returns the payload verbatim.
- Valkey — HSET — the write that stores the payload as the third field.
- Valkey — Documentation — the substrate of record.
Related in this course
- The record hash — the module hub.
- The state and attempts — the previous dive.
- The Protocol — the chapter landing.
- The protocol below the line — the line below which the field set is fixed.
- Data Modeling & Memory (R7) — the data-modeling chapter.