jonnify·redis /redis-patterns/coordination/hash-tag-colocation

R2.05 · Coordination — module hub

Hash-tag co-location.

Force related keys to the same Valkey cluster slot using hash tags, enabling atomic multi-key operations, transactions, and Lua scripts across logically related data. A `{tag}` changes what is hashed — nothing in the data layout moves.

Grounding: echo/apps/echo_mq · EchoMQ.Keyspace.queue_key · content/bcs3.1.md — the hashtag IS the queue name, emq:{orders}:* answers slot 105, slot/1 computed client-side.

The placement explorer — pick a key shape; see the substring Valkey hashes and the real slot. Each key is checked against the reference user:{123}:settings on slot 5970. Hover or select a row.
reference user:{123}:settings → hashes "123" → slot 5970 user:{123}:profilehashes "123" · slot 5970 {user:123}:profilehashes "user:123" · slot 12893 user:123:{profile}hashes "profile" · slot 16237 user:123:profileno tag · whole key key:{}:suffixempty {} · whole key {a}:{b}:keyhashes "a" · slot 15495
readout · Pick a key shape. Only the first non-empty {tag} is hashed; an empty {} or no tag hashes the whole key.

Static verdicts — user:{123}:profile → "123" · slot 5970, co-locates · {user:123}:profile → "user:123" · slot 12893 · user:123:{profile} → "profile" · slot 16237 · key:{}:suffix → whole key · {a}:{b}:key → "a" · slot 15495.

§1

How hash tags work

hash-tag-colocation.md.txt · echo_mq

Valkey cluster distributes keys across 16384 slots by a hash of the key name — so two related keys can land on two nodes, and a multi-key command across them raises CROSSSLOT. A hash tag, a {...} substring, changes what is hashed: Valkey hashes only the tag, so every key carrying the same tag value shares one slot.

Give three keys the same tag value and they all land on one slot:

user:{123}:profile    # hashes "123" → slot 5970
user:{123}:settings   # hashes "123" → slot 5970
user:{123}:sessions   # hashes "123" → slot 5970

An operation across the three runs on one node. Drop the tag and the whole key is hashed instead, scattering the keys to different slots. The slot is CRC16(substring) % 16384 — pure arithmetic over the key bytes, so a client can compute it without asking the server. R2.04 was the fallback for keys that cannot share a slot; this module is the prevention it substitutes for.

A hash tag does not move data; it changes which substring determines the slot, so related keys can be made to share one — the precondition for every multi-key operation.

§2

User data co-location

hash-tag-colocation.md.txt · Basic Pattern

The basic pattern: group all of an entity's keys behind one tag value — the entity id.

user:{user_id}:profile
user:{user_id}:settings
user:{user_id}:notifications
user:{user_id}:sessions

With the keys co-located you can update several fields atomically with MULTI/EXEC, run a Lua script touching all of them, and use WATCH for optimistic locking across them.

One tag value per entity puts all of that entity's keys on one slot — the precondition for every multi-key operation that follows.

§3

Atomic multi-key operations

hash-tag-colocation.md.txt · Atomic ops

A shared slot is what makes each of these legal across the entity's keys. A transaction across related keys:

MULTI
HSET user:{123}:profile name "Alice" updated_at "1706648400"
HSET user:{123}:settings theme "dark"
INCR user:{123}:stats:updates
EXEC

Every command runs atomically because the keys share a slot. A Lua script across keys reads two hashes and writes a third — all co-located, called with three keys on one slot (EVAL <script> 3 user:{123}:profile user:{123}:settings user:{123}:cache). Optimistic locking with WATCH works the same way — watch two co-located keys, read, then MULTI/EXEC; if either changes before EXEC, the transaction aborts.

Transactions, Lua, and WATCH all require every key on one slot — the tag is the precondition each of them rests on.

§4

Real co-location patterns

hash-tag-colocation.md.txt · Patterns

The same move recurs across entity kinds — one tag value per entity, atomic operations across its keys. An order with line items carries the {order_id} tag on its header, items, totals, and status, so an atomic status transition writes the header and the history in one MULTI/EXEC. A counter with inventory shares the {sku} tag across stock, reserved, and sold, so a Lua reserve reads stock and, if enough, DECRBYs stock and INCRBYs reserved in one atomic script. A session with related state shares the {session_id} tag across data, user, cart, and expiry, so session operations stay race-free.

One tag per entity — order, product, session — is the recurring shape: the atomic operation is possible only because the entity's keys all hash together.

§5

Hash tag placement rules

hash-tag-colocation.md.txt · Placement Rules

Where you put the braces determines what is hashed. Three rules, and the explorer at the top of the page is each of them made concrete. The first {...} wins — the first balanced tag determines the slot:

{user:123}:profile    # hashes "user:123" → slot 12893
user:{123}:profile    # hashes "123" → slot 5970
user:123:{profile}    # hashes "profile" → slot 16237 (won't co-locate)

The convention is to put the entity id in the tag. An empty tag {} hashes the whole key (key:{}:suffix → the whole key → slot 14786), so avoid an empty tag unless that is intended. And only the first tag is used — {a}:{b}:key hashes a (slot 15495) and ignores b; two tags do not combine.

First brace, non-empty — or the whole key. Those three rules are the entire placement contract, and every slot above is the real CRC16 over the substring Valkey actually hashes.

§6

The hot-slot problem — and limitations

hash-tag-colocation.md.txt · Hot Slot

Co-location concentrates load. If one entity is extremely hot — a celebrity user, a viral product — its slot becomes a bottleneck. Three mitigations: shard within the entity across tagged sub-keys (user:{123}:followers:0, …:1) so the load splits while the keys stay co-located; separate the hot data onto its own untagged key on a different slot; and accept eventual consistency for read-heavy data by replicating to a non-tagged read replica.

A tag co-locates one entity's keys; it offers no guarantee across entities (a transfer between two users reaches for the cross-shard detection of R2.04), across cluster-wide commands (KEYS/SCAN/FLUSHDB ignore slots), or during resharding (a slot can migrate, and clients retry).

Co-location is a load decision as much as a correctness one — group only what needs to be atomic, and split the rest so no single slot carries a hot entity alone.

§7

Commands that benefit

hash-tag-colocation.md.txt · Commands

Every multi-key command requires its keys on one slot in cluster mode. The tag is what arranges that, so each of these becomes available across an entity's keys:

CommandBenefit
MULTI/EXECAtomic transactions
WATCHOptimistic locking
EVAL/EVALSHALua scripts
MGET/MSETMulti-key read/write
RENAMEAtomic key swap
COPYKey duplication

Notes on ValkeyA Valkey cluster hash tag hashes only the substring in the first non-empty {...}, then takes CRC16 % 16384 — so keys sharing a tag share a slot — valkey.io/topics/cluster-spec.

§8

The pattern, applied — emq:{q}: and client-side slot/1

keyspace.ex · bcs3.1

EchoMQ's queue keys are co-located by exactly this rule, and the grammar makes it a law rather than a convention. EchoMQ.Keyspace.queue_key/2 (echo/apps/echo_mq/lib/echo_mq/keyspace.ex) returns "emq:{" <> q <> "}:" <> type — so the pending set, the active set, the schedule, the dead set, and a job's own row all carry the same {q} tag and answer one slot by construction. The hashtag is the queue name. The slot is computed client-side by slot/1: CRC16-XMODEM (polynomial 0x1021) over the hashtag, modulo 16384, so the connector can route and partition without a server round trip. The functional-Elixir craft behind the echo data layer is taught by the /elixir state chapter.

the application · the keyspace, gated against live Valkey

The hashtag is the queue name — emq:{orders}:* answers one slot

The co-location is not assumed — queue_key/2 always braces the queue name, and slot/1 computes the slot client-side over CRC16 % 16384 (valkey.io/topics/cluster-spec). The committed manuscript record froze the consequence, figure by figure:

content/bcs3.1.md · the co-location law, slot-locality against live Valkeyfrozen
the hashtag IS the queue name
emq:{orders}:* — pending, active, meta, job row   all answer slot 105   ← one queue, one slot
emq:{fills}:*                                  slot 4165
vector  slot("123456789")                      12739   ← the client-side CRC16
§9

The three dives

mechanic → prevention → client-side

Each dive takes the mechanic one layer deeper, in order: the tag rule and the real slot computation first, then why a multi-key script raises CROSSSLOT off one slot and how the {q} tag keeps a queue's claim legal, then how slot/1 computes that slot client-side so the connector routes without a round trip.

the pattern

Hash tags force related keys onto one slot, so a multi-key MULTI/EXEC or Lua script stays legal across them — the prevention that removes the torn write R2.04 detects.

its EchoMQ application

Keyspace.queue_key/2 returns emq:{q}:<type>, so a queue's pending, active, and job-row keys all share the {q} tag and answer one slot; slot/1 computes that slot client-side over CRC16 % 16384, so routing needs no round trip.

Three dives, one mechanic: the tag selects the hashed substring, colocation keeps a multi-key command legal, and slot/1 computes the slot on the client.

After R2.05, the Coordination chapter closes with the workshop, which makes a codemojex guess submission atomic across runtimes on these co-located emq:{cm}:* keys. Return to the Coordination chapter to continue, or open the Overview for where Valkey sits under codemojex.

§10

References

sources · related

Sources

Related in this course