R6.01.3 · Rate limiting — dive

Global, not local.

A rate limit is only worth as much as the place its counter lives. Put the counter in a worker process and the budget is local to that worker; put it in Valkey and the budget is one number every worker shares. The whole value of a server-side limiter is that the budget is global by construction — it cannot scale with the fleet, because there is only one counter to spend.

Grounding: echo/apps/echo_mq/lib/echo_mq/{keyspace,metrics}.ex · the emq:{q}:limiter counter, the emq:{q}:meta cap read by EchoMQ.Metrics.get_global_rate_limit/2, the slot pinned by EchoMQ.Keyspace.slot/1.

One shared counter, or N private ones. Select where the counter lives to read the budget the fleet actually enforces. The same admit trace runs through both; only the placement differs.

readout · Select a placement. Shared — four workers INCR one server-side counter, so the fleet holds at max. Local — four private counters each reach max, so the fleet admits about 4×max.

§1

Where the counter lives

metrics.ex · get_global_rate_limit/2

A limiter has two parts: a counter that is spent down, and a cap the counter is measured against. In EchoMQ both are queue keys, and neither lives in a worker. The counter is emq:{q}:limiter — a plain string a worker INCRs on each admit. The cap is the max field of emq:{q}:meta, a HASH, written once when the queue is configured and read by every worker.

# EchoMQ.Metrics — the cap is read, never owned. One HGET, one number per queue.
def get_global_rate_limit(conn, queue) do
  # HGET emq:{q}:meta max — the queue's configured cap, 0 when unconfigured
  case Connector.command(conn, ["HGET", Keyspace.queue_key(queue, "meta"), "max"]) do
    {:ok, max} -> {:ok, to_int(max)}
    other -> other
  end
end

That placement is the entire point. A worker does not own its slice of the budget; it reads the shared cap and spends the shared counter. Ten workers, three runtimes, four nodes — they INCR the same key against the same max. The budget is a property of the queue, held in Valkey, not a property of any process. A worker can restart, a node can be added, a second runtime can join, and the budget does not change, because none of them carried a piece of it.

The contrast names what server-side buys: a server-side counter is shared state with atomic arithmetic. INCR returns the post-increment value in one round trip, so two workers incrementing at the same instant read two different values — there is no read-modify-write race to lose a count to. The budget is exact across the fleet because the fleet shares one atomic counter.

Two parts, both in Valkey: the emq:{q}:limiter counter and the emq:{q}:meta max. A worker reads the cap with get_global_rate_limit/2 and spends the counter — it owns neither, so the budget is the queue's, not the process's.

§2

The N× overshoot of a local limiter

the silent failure server-side avoids

The failure a server-side counter avoids is the local limiter — a limiter whose counter lives in each worker process. It looks correct in a single-worker test: count admits, refuse past max, reset on the window. It is correct, for one worker. The defect appears the moment a second worker starts.

With N workers each enforcing max against its own private counter, the system admits up to N × max per window. Each worker is individually within budget; the fleet is N times over it. Worse, the overshoot is silent and it tracks the fleet — scale from two workers to ten to handle load and the effective rate scales with them, from 2×max to 10×max, with nothing in the limiter to signal it. The number you configured is the per-worker rate, not the system rate, and the gap between them is exactly the worker count.

A shared server-side counter has no such gap. The rate is max, full stop, because there is one counter and max is measured against it regardless of how many workers spend it. This is why the limiter counter belongs in Valkey: not for speed, but because shared is the only place a budget can be global. A budget that is global by construction cannot drift as the fleet grows.

Counter placement	Per-worker view	Fleet of N workers admits
Shared — `emq:{q}:limiter` in Valkey	within `max`	`max` — exact, fleet-wide
Local — a counter per worker process	within `max`	`N × max` — silent, scales with the fleet

the pattern

A rate limit is the cap on a shared resource, so its counter must be shared too. A counter held in each worker enforces the cap N times over — once per worker — and the system admits N× the rate, silently, scaling with the fleet.

its EchoMQ application

The budget is one server-side counter, emq:{q}:limiter, spent against the queue's one max. Because the counter lives in Valkey and not in a worker, the budget is global by construction: every worker spends the same number, so the rate stays max no matter how many workers run.

A local limiter is correct for one worker and wrong for a fleet: N workers each within max sum to N×max. The shared counter holds the line because there is only one to spend.

§3

One key, one slot

keyspace.ex · slot/1 · hashtag/1

A budget that is one counter raises one question: with the keys spread across a Valkey cluster, do all of a queue's workers reach the same counter? They do, and the mechanism is the {q} hashtag. Valkey Cluster shards the keyspace into 16384 hash slots, and a key's slot is computed from the substring inside its first {...} braces when one is present — so every key of a queue carries the per-queue hashtag and they all hash to one slot.

EchoMQ computes the slot client-side, so the connector can route and partition without a server round trip.

echo/apps/echo_mq/lib/echo_mq/keyspace.ex · EchoMQ.Keyspace · the slot is CRC16-XMODEM over the hashtagverbatim

@spec slot(binary()) :: 0..16383
def slot(key) when is_binary(key), do: rem(crc16(hashtag(key), 0), 16384)

@spec hashtag(binary()) :: binary()
def hashtag(key) do
  with [_, rest] <- :binary.split(key, "{"),
       [tag, _] when tag != "" <- :binary.split(rest, "}") do
    tag                                        ← the substring inside the first {...}
  else
    _ -> key
  end
end                                            ← vector: slot("123456789") == 12739

For queue q, both emq:{q}:limiter and emq:{q}:meta share the hashtag q, so they land on the same slot — and so does every other key of that queue. The consequence is twofold. First, every worker's INCR emq:{q}:limiter reaches one key on one slot — the counter is genuinely shared, atomic, not split per shard. Second, a multi-key Lua script over a queue's keys is legal, because all its keys are on one slot — there is no CROSSSLOT error to hit. The hashtag is what makes the limiter both global and co-located: one key, one slot, every worker's increment landing on it.

Every key of the queue, one slot. Select a key to see where it hashes. They all carry the {q} hashtag, so they all land on the same slot — the counter every worker reaches is one key.

readout · Select a key. All three carry the {q} hashtag, so slot/1 hashes the brace bytes alone — every key of the queue lands on one slot, and the limiter the workers share is one atomic key.

Notes on ValkeyValkey Cluster maps the keyspace to 16384 hash slots and computes a key's slot from the bytes inside the first {...} when present — so the {q} hashtag forces every key of a queue onto one slot, which is what keeps a multi-key script free of CROSSSLOT and the counter atomic — valkey.io/topics/cluster-spec.

§4

The pattern, applied

echo/apps/codemojex

The pattern lands on the shipped surface. A server-side counter spent against a shared cap is exactly the emq:{q}:limiter / emq:{q}:meta max pair: one number every worker INCRs, one cap every worker reads with get_global_rate_limit/2, both pinned by the {q} hashtag to one slot. Wiring the increment into the claim path — checking the counter as each job is claimed and deferring a job that is over budget — is EchoMQ's scaling layer, taught in depth in the EchoMQ course.

The worked consumer is codemojex (echo/apps/codemojex), a Telegram emoji-guessing game on the same stack and a multi-bot deployment by design. Its guesses ride one queue: Codemojex.Guesses.submit/3 enqueues a branded JOB on the player's lane on the cm queue, so the keyspace is emq:{cm}:. A budget that capped guesses for that queue would be the emq:{cm}:limiter counter — one number shared across every bot worker, global because it is server-side, co-located because every key of the cm queue carries the {cm} hashtag onto one slot.

codemojex also shows the other half of the lesson honestly. Its outbound Telegram throttle, Codemojex.RateLimiter, is a process-local token bucket — and that is the right tool there, because the budget it enforces is one node's outbound API allowance to Telegram, not a budget the whole fleet must share. Local is correct when the resource is local; global is required when the resource is shared. The two limiters live in different places because they bound different things.

/echomq/queue

References

sources · related

Sources

Valkey — Cluster specification — the 16384 hash slots and the hashtag rule that pins a queue's keys to one slot.
Valkey — INCR — the atomic increment a shared counter is spent down with; returns the post-increment value in one round trip.
Valkey — CLUSTER KEYSLOT — the server-side counterpart of the client-side slot/1, for verifying a key's slot.
Stripe — Scaling your API with rate limiters — a production account of why limiter state is shared infrastructure, not per-process.

Related in this course

R6.01 · Rate limitingthe module hub — the four algorithms and the EchoMQ limiter
R6.01.1 · Fixed & sliding windowsthe window algorithm the shared counter runs
/echomq/queuethe scaling layer: the limiter wired at the claim point
/bcs/busthe Valkey-native bus and the emq:{q}: keyspace
/bcs/codemojexthe multi-bot consumer whose guess queue the counter would bound