jonnify·redis /redis-patterns/coordination/hash-tag-colocation/cluster-auto-detect

R2.05.3 · Hash-tag co-location — dive 3

Cluster auto-detect.

Compute a key's slot on the client, ahead of any command — so the connector routes every key to its node without a server round trip.

Grounding: echo/apps/echo_mq/lib/echo_mq/keyspace.ex · slot/1 = rem(crc16(hashtag(key), 0), 16384), CRC16-XMODEM computed client-side · vector slot("123456789") == 12739 asserted in the module.

The client-side slot table — a queue's six keys, each slot computed by slot/1. Toggle the queue's {tag}; hover or select a key row. With the tag, all six answer one slot; without it, they scatter.
slot/1 · emq:{orders}:* · 6 keys 6 keys → 1 slot (105) — the queue lives on one slot
readout · Toggle the {orders} tag, then pick a key. slot/1 computes each slot client-side; with the tag all six answer slot 105.
§1

The slot is pure arithmetic

hash-tag-colocation.md.txt · Design guidelines

A key's cluster slot is CRC16(substring) % 16384 over the part the cluster hashes. That formula is deterministic: given the key bytes, the slot is fixed. So a client does not need to ask the server — it can compute the slot itself and reach the same number the cluster would assign. That is what makes client-side routing possible: a cluster-aware client maps a key to its slot, the slot to its node, and sends the command straight to the owner, with no detour through a wrong node and a redirect.

The live server-side counterparts confirm the same arithmetic. CLUSTER KEYSLOT <key> returns a key's slot; CLUSTER SLOTS returns the slot-to-node map. A client that computes the slot itself can skip the first round trip entirely and consult only the cached slot map for the second.

The slot is a function of the key bytes, not a property the server has to be asked — so a client can compute it and route ahead of the command.

§2

EchoMQ computes the slot client-side

keyspace.ex · slot/1 + crc16/2

EchoMQ.Keyspace.slot/1 (echo/apps/echo_mq/lib/echo_mq/keyspace.ex) is exactly that client-side computation:

# echo/apps/echo_mq/lib/echo_mq/keyspace.ex — the cluster slot, computed client-side
def slot(key) when is_binary(key), do: rem(crc16(hashtag(key), 0), 16384)

defp crc16(<<>>, crc), do: crc
defp crc16(<<b, rest::binary>>, crc) do
  crc = Bitwise.band(Bitwise.bxor(crc, Bitwise.bsl(b, 8)), 0xFFFF)
  crc =
    Enum.reduce(1..8, crc, fn _, c ->
      shifted = Bitwise.band(Bitwise.bsl(c, 1), 0xFFFF)
      if Bitwise.band(c, 0x8000) != 0, do: Bitwise.bxor(shifted, 0x1021), else: shifted
    end)
  crc16(rest, crc)
end

crc16/2 is a real inline Elixir XMODEM with polynomial 0x1021 — the cluster specification's algorithm, byte for byte. The module's own moduledoc states the intent: "Cluster slot of a key, computed client-side … so the connector can route and partition without a server round trip. Known vector: slot("123456789") == 12739 (CRC16 0x31C3)." The vector is the contract: a client that produces 12739 for 123456789 is running the same arithmetic the cluster runs. Because queue_key/2 always braces the queue name, a queue's keys answer one slot, and the client resolves it locally rather than over the wire.

Notes on ValkeyCLUSTER KEYSLOT returns a key's slot and CLUSTER SLOTS the slot-to-node map — the same CRC16 % 16384 a client can run itself to route without the round trip — valkey.io/commands/cluster-slots.

§3

The two lives of the slot function

hash-tag-colocation.md.txt · Limitations

Today EchoMQ's topology is a single Valkey instance. On one node there is one keyspace and no slot boundary, so multi-key commands are always legal and the slot is never consulted for routing. The function is computed, asserted against its vector, and parked. On the clustered day the same function becomes routing: the co-location law — every per-queue key wraps the queue name in {q} — means each queue's keys answer one slot, so the per-queue transition scripts written today (claim moves a job between pending and active; retry touches the job row and a schedule) stay single-slot legal on the clustered day with no edit. Pick a side of the figure to read each life.

Two lives of slot/1 — partition arithmetic now, routing on the clustered day. Hover or select a panel.
slot/1 · the same function, two lives single instanceone keyspaceno slot boundaryslot/1: computed,asserted, parkedmulti-key: always legal clusterkeyspace across slotsco-location law holdsslot/1: routingper-queue scripts:single-slot, no edit
readout · Pick a life. Today slot/1 is parked partition arithmetic; on the clustered day it becomes routing, and the co-location law carries the per-queue scripts over with no edit.
today

One keyspace, no slots. A multi-key move is always legal, and slot/1 is computed, asserted against its vector, and parked — partition arithmetic the connector can run without a round trip.

the clustered day

The same slot/1 becomes routing. The co-location law means each queue's keys answer one slot, so the per-queue transition scripts written today survive the move with no edit.

The transferable pattern: compute a key's slot on the client from the same CRC16 % 16384, so routing and co-location checks cost no round trip and a cross-slot layout is visible before a command is sent.

§4

In EchoMQ — the slot, computed and frozen

keyspace.ex · bcsA

Because queue_key/2 always braces the queue name, a queue's keys answer one slot, and the client resolves it locally rather than over the wire. The manuscript froze the figures against the live wire: "the slot law held client-side — emq:{orders}:* mapped to slot 105 == 105 against 8507 for the payments queue, with the specification vector answering 12739." Two queues, two slots, computed entirely on the client.

the pattern

Hash-tag co-location lands related keys on one slot, and the slot is CRC16(substring) % 16384 — pure arithmetic a client can run itself.

in EchoMQ

slot/1 computes that slot client-side from the {q} hashtag (CRC16 % 16384), so the connector routes and partitions without a server round trip — with the vector slot("123456789") == 12739 asserted.

The auto-detect is the client-side slot computation: slot/1 runs the cluster's own arithmetic locally, so a queue's co-location is known before a command is sent, not discovered after one is refused.

the application · the slot, frozen against live Valkey

Two queues, two slots, computed on the client

The slot is not assumed — slot/1 runs CRC16-XMODEM client-side over the hashtag, mod 16384, the cluster specification's algorithm (valkey.io/topics/cluster-spec). The committed connector record froze it, figure by figure:

content/bcsA.md · the slot law, computed client-side against live Valkeyfrozen
emq:{orders}:*                slot 105 == 105   ← the orders queue, one slot
emq:{payments}:*              slot 8507      ← a second queue, a second slot
specification vector          12739         ← slot("123456789")

The closed emq:{q}:<type> grammar, the {emq}: reserve, and the version fence the connector negotiates before the first command are the subject of the dedicated EchoMQ course and the BCS keyspace chapter.

R2.05 ends here; the chapter closes with the R2.06 workshop, where the coordination patterns combine on a real codemojex guess submission.

§5

References

sources · related

Sources

  • Valkey — Cluster specification — the 16384-slot model, hash tags, and the same-slot rule the client-side slot/1 computes ahead of time.
  • Valkey — CLUSTER SLOTS — the command that returns a cluster's slot-to-node map; the cached map a client consults after computing the slot itself.
  • Valkey — CLUSTER KEYSLOT — the command that returns a key's slot, the server-side equivalent of the client-side slot/1.

Related in this course