jonnifyechomq
EchoMQ / The Protocol / The owned keyspace / The hashtag & the slot

The owned keyspace · dive 02 · the slot

The hashtag & the slot

The brace is the routing key. The slot is one CRC16 of it, the same in every client and on every node.

The braces in emq:{q}:<type> are the cluster hashtag. A Valkey cluster routes a key by a CRC16 of the substring inside its first {...} — so every key that shares a hashtag shares a slot. EchoMQ.Keyspace computes that slot client-side, with the cluster specification's exact algorithm: CRC16-XMODEM over the hashtag, modulo 16384, pinned by the vector slot("123456789") == 12739. The scripts rely on that one slot per queue for cluster legality; the native module serves standalone servers, where every key is local.

16384 slots · one queue lands on one

0 4096 8192 12288 105 {orders}
queue orders → slot/1 = crc16("orders") % 16384 = 105 — every key emq:{orders}:* lands on slot 105, so the queue is single-slot

Why one slot matters

A Valkey cluster splits the keyspace into 16384 slots and assigns each slot to a shard. It routes a key by a CRC16 of the substring inside its first {...} — not the whole key. So every key that shares a hashtag is guaranteed to live on the same shard. By wrapping the queue name in braces, EchoMQ makes one queue's keys all co-resident: emq:{orders}:pending, emq:{orders}:active, and emq:{orders}:job:<id> all hash on orders, so they land together.

What that one slot buys depends on the executor that runs the transition.

the scripts
rely on it for cluster legality. A cluster runs a multi-key script only when its keys hash to one slot; declared keys that span slots are refused with CROSSSLOT. A script's declared keys share {q}, and the row @claim2 derives from the declared base KEYS[3] carries the same {q}, so it lands on the same slot. The connector reads {emq}:version once per connect, as a plain GET outside any script, so no script touches the reserve's slot.
the native module
serves standalone servers only. On a standalone server every key is local, so no slot can split a command. The canon records the limit: every native write verb also reads {emq}:version, whose hashtag is emq, not the queue's, so in general it sits on another slot. The module keeps the hashtag discipline anyway: its layout check refuses any call whose keys do not share one non-empty {q}, so its keys are the scripts' keys, byte for byte. A cluster-enabled server refuses every EMQ.* write with EMQWIRE expected a standalone server but cluster mode is enabled.

Without the brace, three keys of one queue could scatter across three slots, and on a cluster no single atomic script could span them.

hashtag/1 — the substring inside the first braces

The slot is computed over the hashtag, so the first job is to extract it. hashtag/1 returns the substring inside the first {...}, and falls back to the whole key when the braces are absent or empty — that fallback is the cluster rule, not a convenience.

# echo_mq — EchoMQ.Keyspace
# The hashtag is the substring inside the first {...}. When the braces are
# absent, or empty, the whole key is hashed — the cluster rule. For an
# EchoMQ key the tag is always the queue name (emq:{orders}:* -> "orders").
def hashtag(key) do
  with [_, rest] <- :binary.split(key, "{"),
       [tag, _] when tag != "" <- :binary.split(rest, "}") do
    tag
  else
    _ -> key
  end
end

For an EchoMQ key the tag is the queue name, because the grammar puts the queue — and only the queue — inside the braces (an empty name is the one exception, below). So hashtag("emq:{orders}:pending") is "orders", and computing the slot of the full key gives exactly the same answer as computing it of the bare queue name. The brace is the routing key; the rest of the string does not move it.

slot/1 — CRC16-XMODEM, modulo 16384

slot/1 is the hashtag fed through CRC16-XMODEM, modulo 16384: the slot a cluster server assigns, computed client-side, so the slot of a key is knowable before the key is ever sent. The connector does not route by it; it holds one connection to one server. The conformance suite uses it to check where keys land.

# echo_mq — EchoMQ.Keyspace
# The cluster slot of a key, computed client-side: the slot a cluster
# server assigns. CRC16-XMODEM over the hashtag (not the whole key),
# modulo 16384 — the cluster specification's algorithm. Known vector:
# slot("123456789") == 12739 (CRC16 0x31C3).
def slot(key) when is_binary(key), do: rem(crc16(hashtag(key), 0), 16384)

The CRC16 itself is the standard cluster variant — polynomial 0x1021, initial value 0, no reflection. One byte at a time: XOR the byte into the high half of the running value, then eight rounds of shift-and-conditional-XOR, masked to 16 bits.

# echo_mq — EchoMQ.Keyspace
# CRC16-XMODEM: poly 0x1021, init 0, no reflection. XOR the byte into the
# high half, then eight shift-and-conditional-XOR rounds, masked to 16 bits.
defp crc16(<<>>, crc), do: crc

defp crc16(<<b, rest::binary>>, crc) do
  crc = Bitwise.band(Bitwise.bxor(crc, Bitwise.bsl(b, 8)), 0xFFFF)

  crc =
    Enum.reduce(1..8, crc, fn _, c ->
      shifted = Bitwise.band(Bitwise.bsl(c, 1), 0xFFFF)
      if Bitwise.band(c, 0x8000) != 0, do: Bitwise.bxor(shifted, 0x1021), else: shifted
    end)

  crc16(rest, crc)
end

The vector pins the implementation: slot("123456789") == 12739. This is what makes the slot part of the protocol. The server alone decides a key's slot, so a speaker that computes slots — a cluster-aware client, a test — must get the server's answer for the same hashtag, or it sends commands to the wrong node and is redirected; the keys themselves never move. The algorithm and its constants come from the cluster specification, not an implementation choice.

The native module carries its own port of the three functions. Its tests assert the same vector and replay golden vectors generated from the Elixir functions, so both executors compute the same hashtag and the same slot for every key:

// trafikey — the native keyspace mirror
// The same rule as hashtag/1: the bytes between the first "{" and the
// next "}", when non-empty; otherwise the whole key.
pub fn hashtag(key: &[u8]) -> &[u8] {
    let mut opened = key.splitn(2, |&b| b == b'{');
    let (Some(_), Some(rest)) = (opened.next(), opened.next()) else {
        return key;
    };
    let mut closed = rest.splitn(2, |&b| b == b'}');
    match (closed.next(), closed.next()) {
        (Some(tag), Some(_)) if !tag.is_empty() => tag,
        _ => key,
    }
}

// CRC16-XMODEM, poly 0x1021, init 0: the same loop on a u16, whose shift
// drops the high bit the Elixir masks away.
pub fn crc16(bytes: &[u8]) -> u16 {
    let mut crc: u16 = 0;
    for &b in bytes {
        crc ^= u16::from(b) << 8;
        for _ in 0..8 {
            crc = if crc & 0x8000 != 0 {
                (crc << 1) ^ 0x1021
            } else {
                crc << 1
            };
        }
    }
    crc
}

// slot/1, ported. The module's test pins the vector:
// assert_eq!(slot(b"123456789"), 12739);
pub fn slot(key: &[u8]) -> u16 {
    crc16(hashtag(key)) % 16384
}

The empty hashtag — the one trap

The hashtag rule has one trap, and the native layout check closes it. An empty queue name gives emq:{}:<type>, whose braces hold nothing, so the whole key is hashed: one queue spread over several slots. The host builder composes such keys and the scripts run them on a standalone server; the native module refuses an empty name outright, before it reads anything.

emq:{}:pending
slot 7387 — the whole key is hashed.
emq:{}:active
slot 5857 — another slot for the same queue.
emq:{orders}:pending
slot 105 — and every other key of orders with it.

Compute a slot — and see both stances

Type a queue name. The page composes the queue's keys and the reserve key, and runs the same hashtag/1 and slot/1 over each, over the UTF-8 bytes as the byte-wise Elixir does. It prints every key's slot, then the two stances: the scripts' declared keys, and a native write's keys plus its fence read of {emq}:version. The check chip 123456789 must read 12739: that is the vector that proves the port. Try emq, and the empty name.

slot/1 · type a queue name

hashtag(emq:{orders}:pending) = orders
crc16 = 105 (0x0069) → % 16384 → slot = 105
emq:{orders}:pending → 105 · emq:{orders}:active → 105 · emq:{orders}:job:JOB0Nb1VTbfnu4 → 105
{emq}:version → 9051 — the reserve, hashed on emq
scripts · declared keys on slot 105 — one slot: legal on a cluster; the fence GET runs at connect and on every reconnect, outside any script
native · a write's keys on slot 105, its fence read of {emq}:version on slot 9051 — two slots: the module serves standalone servers, where every key is local

The slot is one CRC16 of the hashtag, knowable before the wire is touched. One queue, one hashtag, one slot: the precondition the scripts carry onto a cluster. The native module serves standalone servers, where every key is local, and still refuses keys that do not share one hashtag.

Pattern → implementation

the pattern (Redis Patterns Applied)

Hash-tag co-location: wrap the shared part of a key in {...} so a group of keys routes to one slot, where an atomic multi-key operation is legal. Redis Patterns Applied — the tag mechanic shows the slot computation, and CROSSSLOT prevention the refusal it prevents.

the implementation (echo_mq + the native module)

EchoMQ.Keyspace.slot/1 computes that slot client-side from the queue hashtag — CRC16 modulo 16384 — the same slot the server assigns, and every key a script touches carries that hashtag, so each script runs on one slot. The native module serves standalone servers, where no slot can split a call, and still refuses keys that do not share one {q}.

The pattern says "co-locate by hashtag"; slot/1 is the exact, pinned arithmetic that makes the co-location real and the same for every speaker, in either executor.

Recap — the brace, the CRC16, the slot

The {q} brace is the cluster hashtag. hashtag/1 extracts the substring inside the first {...}; slot/1 runs it through CRC16-XMODEM modulo 16384, client-side, pinned by slot("123456789") == 12739 and ported to the native module with the same vector. One queue hashes to one slot: the scripts rely on it for cluster legality, while the native module serves standalone servers, where every key is local, and refuses an empty {}. The next dive reads the space that sits beside every queue: the reserve.

Next in this module: The reserve — {emq}:, the core's first-byte-disjoint cross-queue space, where the wire identity lives.

References

Where the slot algorithm and the hashtag routing rule are specified.

Sources

Related in this course