EchoMQ / The Protocol / Immutability & branded ids / The branded-id gate

Immutability · dive 02 · how

The branded-id gate

A job's id is a 14-byte branded Snowflake. The host checks all fourteen bytes; the wire checks the namespace before a row exists.

A caller does not pick a job's id. It is a 14-byte branded id: a three-letter uppercase namespace, then eleven characters of Base62 Snowflake. For a job the namespace is JOB. The host checks the whole id before any wire: Keyspace.job_key/2 raises unless all fourteen bytes parse. The transition checks the namespace before any write: the @enqueue script reads the first three bytes and refuses anything that is not JOB, with the EMQKIND error class, before EXISTS, before HSET — and the native EMQ.ENQUEUE runs the same three-byte check with the same refusal. Decode a real id on the right, then read the gates that admit it.

Decode a 14-byte id The gate, in three beats The gate, byte by byte Pattern & implementation References

The 14-byte branded id · pick one to decode

id JOB0KHTOWnGLuC (14 bytes)
namespace: JOB · snowflake: 274557032793636864
ts: 2026-01-27 15:11:37 UTC · node: 0 · seq: 0
host (job_key/2 → valid?/1): passes — all fourteen bytes parse
wire gate: admitted — the namespace is JOB, in the script and in EMQ.ENQUEUE alike.

Identity is not a number the caller picks

A job's id is a 14-byte branded id: a three-character uppercase namespace concatenated with an eleven-character Base62 encoding of a 63-bit Snowflake. The layout is fixed — three plus eleven, always fourteen — and the Snowflake itself packs a 41-bit timestamp, a 10-bit node, and a 12-bit sequence (ts << 22 | node << 12 | seq) against a fixed epoch (1704067200000). For a job, the namespace is JOB.

The name carries the weight a type used to — four properties, and the gate reads the first of them. It is typed: the namespace declares the domain class on the wire, so a USR id cannot enter the job keyspace. It is ordered: the text sorts as its mint instant sorts, so a table keyed by it is a timeline. It is placed: one function, EchoData.BrandedId.hash32/1, places a row from the identity's Snowflake alone — the Snowflake of USR0KHTOWnGLuC hashes to 234878118, the same for every holder. And it is conformant: one source of truth makes encode and decode identical across the runtimes, so which language speaks the wire is a deployment detail.

The decoder above takes a real id apart: the namespace, the Snowflake, and the unpacked timestamp, node, and sequence — and the two verdicts. A JOB id passes the host's 14-byte check and the namespace gate; a USR id is the right shape, so the host passes it, but the wrong namespace, so the gate refuses it; a bare "job-42" is not a branded id at all, and the host raises before any wire. The branding is not a label — it is what the transition tests before the id may enter the job keyspace.

These are not aspirations: the runtime asserts them at boot. self_check!/0 in echo_data's branded_id.ex compares the native and pure code paths against the contract's vectors and refuses to start if they disagree — so the name is trustworthy because the system proves it before trusting it. The three lines below state three of its four vectors — source truths, not measurements; the fourth encodes the same Snowflake back to USR0KHTOWnGLuC.

echo_data · branded_id.ex · the vectors `self_check!` asserts at boot

hash32(274557032793636864)  ->  234878118   -- the Snowflake of USR0KHTOWnGLuC; native and pure agree
parse("USR0NgWEfAEJfs")     ->  {:ok, "USR", 320636799581945856}
decode("USRzzzzzzzzzzz")    ->  :error      -- an overflow is refused, not wrapped

The gate, in three beats

Beat one — the named handle. EchoMQ.Jobs.enqueue/4 gates the queue name, builds the declared keys — the row key through job_key/2, so the host's 14-byte check runs here — and runs the @enqueue script through EchoMQ.Connector, unless the optional :via picks another dispatch module. Then it maps the server's verdict back to a typed result, including the EMQKIND refusal the gate produces, which becomes {:error, :kind}.

# echo_mq — EchoMQ.Jobs
# valid_queue!/1 refuses a brace or NUL in the queue name; job_key/2 raises
# unless the id is a valid 14-byte branded id (the host's full check). The
# server's verdict is then mapped: 1 enqueued, 0 a duplicate, and an EMQKIND
# error reply (the namespace gate firing) becomes the typed {:error, :kind}.
def enqueue(conn, queue, job_id, payload, opts \\ [])
    when is_binary(job_id) and is_binary(payload) do
  Keyspace.valid_queue!(queue)
  via = Keyword.get(opts, :via, Connector)
  keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")]

  case via.eval(conn, @enqueue, keys, [job_id, payload]) do
    {:ok, 1} -> {:ok, :enqueued}
    {:ok, 0} -> {:ok, :duplicate}
    {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind}
    other -> other
  end
end

Beat two — the gate, in the script body. The first statement of @enqueue is the namespace check. ARGV[1] is the job id; string.sub(ARGV[1], 1, 3) is its first three bytes, and the script reads no others. Not 'JOB' → redis.error_reply('EMQKIND …'), and the script stops before the idempotency check, before the row write, before the pending insert.

-- the @enqueue script — the namespace gate is the first statement
-- ARGV[1] is the job id. The script reads only its first three bytes,
-- the namespace; the other eleven are the host's to check. A job id must
-- be JOB-namespaced; anything else is refused before a row exists.
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
-- only a JOB-namespaced id reaches the idempotency check and the row write
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

Beat three — the same gate, natively. EMQ.ENQUEUE 2 emq:{q}:job:<id> emq:{q}:pending <id> <payload> runs the same check inside the server. The order differs: the native handler first checks the call's shape, then that KEYS[1] is exactly emq:{<q>}:job: + the id, then the fence, and only then the namespace; the script checks the namespace first. The verdict on the id is the same, and so is the refusal text, byte for byte.

// trafikey — keyspace::is_job_id, the kind door EMQ.ENQUEUE calls
// Exactly the Lua test: the first three bytes are JOB. A shorter id has
// a shorter prefix, so "JO" fails here as string.sub fails it in Lua.
pub fn is_job_id(id: &[u8]) -> bool {
    id.starts_with(b"JOB")
}

// trafikey — errors::KIND_JOB, the refusal, byte-exact with the script's
pub const KIND_JOB: &str = "EMQKIND job id must be JOB-namespaced";

So the id is checked twice, by two owners, for two different things. The host checks the whole id: EchoMQ.Keyspace.job_key/2 calls EchoData.BrandedId.valid?/1 and raises ArgumentError unless all fourteen bytes parse — three uppercase letters, then eleven Base62 characters that decode without overflow — so a malformed id, a caller's programming error, never reaches the wire. The transition checks the namespace: a well-formed id in the wrong namespace is runtime data, refused on the server as a typed verdict. The wire gate is exactly three bytes, in the script and in the native command alike: an id that begins with JOB but is not a branded id (JOB-42) passes both, and only the host's check refuses it.

A job id is a 14-byte branded Snowflake under JOB. The host checks all fourteen bytes; the transition — the first line of @enqueue, the same test inside EMQ.ENQUEUE — compares three, the wrong namespace refused with EMQKIND before any write.

The gate, byte by byte

The wire gate is one comparison: the first three bytes against 'JOB' — string.sub(id, 1, 3) in the script, is_job_id in the native command. The host's check is a separate one, over all fourteen bytes. Pick a candidate id and read the exact bytes the wire gate reads, its verdict on both executors, and what the host does with the same id.

string.sub(id, 1, 3) ~= 'JOB' · pick a candidate

string.sub("JOB0KHTOWnGLuC", 1, 3) = JOB == 'JOB' → admitted by the script, and by EMQ.ENQUEUE’s is_job_id; the transition proceeds to the row write.
host: passes — a valid 14-byte branded id; the Elixir path reaches the wire and the row is written.

the pattern (Redis Patterns Applied)

A branded key carries its type in its bytes, and the atomic step that touches it can check that type before it acts. Coordination teaches the atomic check-then-act move the gate is an instance of.

the implementation (echo_mq + trafikey)

@enqueue's first statement is string.sub(ARGV[1], 1, 3) ~= 'JOB' → EMQKIND; EMQ.ENQUEUE runs the same three-byte check (is_job_id); Keyspace.job_key/2 checks all fourteen bytes host-side via EchoData.BrandedId.valid?/1.

The namespace gate lives in the transition, not in one runtime's host code, so it holds for every speaker and both executors; the full 14-byte check is the host's.

Recap — the gate and the id

A job id is a 14-byte branded Snowflake under JOB: EchoData.BrandedId fixes the layout (a three-character namespace + eleven Base62 characters) and gives it four properties — typed, ordered, placed (hash32) and conformant, the contract self_check!/0 asserts at boot. Keyspace.job_key/2 checks all fourteen bytes host-side before any wire. The @enqueue script's first line — string.sub(ARGV[1], 1, 3) ~= 'JOB' — and the same test inside EMQ.ENQUEUE read the typed property to refuse the wrong namespace with EMQKIND before any write; neither reads the other eleven bytes. The next dive reads the third guard — how an executor is fenced off a node whose wire it does not speak.

Next in this module: The version fence — the {emq}:version key the connector claims or verifies at connect on its default wire 4, and the native module verifies on every write.

References

Where the branded-id contract and the scripting that enforces it are documented.

Sources

Redis — EVALSHA — the load-once, run-by-SHA dispatch that runs the scripted gate atomically inside the engine.
Valkey — Modules Introduction — how the native module adds EMQ.ENQUEUE to the server.
Valkey — Documentation — the substrate of record, and its server-side Lua semantics.
Valkey — HSET — the row write that only a gated id reaches.

Related in this course

The immutable line — the previous dive: the fixed line.
The version fence — the next dive: the node's fence.
The Branded Component System — where the 14-byte branded id is the identity contract.