R6 · Flow Control & Scale
Staying stable under load.
A queue that runs and orders work has not yet learned to survive a flood. Flow control is the set of moves that keep the system stable under load — bound throughput to a budget, keep tenants fair, enqueue in bulk, and plan worker capacity — all without losing correctness.
The grounding is EchoMQ's real, shipped flow-control layer (echo/apps/echo_mq),
backed by Valkey. The rate limiter spends down a counter key (emq:{q}:limiter) against a
configured max over its own window, read with EchoMQ.Metrics.get_rate_limit_ttl/3;
the fair-lane ring in EchoMQ.Lanes rotates one step per claim so every lane is served in turn;
and EchoMQ.Jobs.enqueue_many/4 admits many jobs in one wire flush. Worked through
codemojex's traffic. This chapter keeps the work of R3–R5 stable when demand exceeds what one
moment can absorb.
Why & when
echo/apps/echo_mqA queue that distributes and orders work still trusts every caller to be well-behaved. Under load that trust breaks: one tenant floods the engine, a burst of writes outpaces the workers, a thousand small enqueues each pay a full round trip. Flow control is how the system stays correct and stable when demand exceeds what one moment can absorb. Each demand below has one matching technique, and the chapter is the set of those answers.
Throughput must stay under a budget
Cap how much work is admitted per unit of time — count what is admitted in the current window, refuse or defer once the count reaches the limit, and reset when the window rolls. The budget lives in the engine, not in any one worker.
Every tenant gets a fair turn
Under contention, no single tenant should starve the rest — a rota serves every lane in turn, and a weight tilts the share without ever letting a lane jump the line. Fairness is constructed, not hashed.
Bulk work must not pay per-item round trips
Admitting a thousand jobs one wire call at a time is a thousand round trips — one flush admits them all, in input order, under the same idempotency as a single add.
Worker capacity must be planned, not guessed
Each claim costs a fetch from the engine — concurrency is bounded by that per-claim cost and by the per-lane ceiling, so capacity is planned against the real fetch rather than assumed.
max, the ring rotates one step per claim.Every technique in this chapter answers one demand load makes on the queue — bound the throughput, keep tenants fair, enqueue in bulk, and plan capacity — each one a real move in EchoMQ's flow-control layer.
The patterns
six modules · one builtOne teaching module is built; the rest are specified. Rate limiting is the one new catalog pattern here; the others are flow-control techniques that compose primitives from earlier chapters. Each module is a hub with its dives, grounded in the real EchoMQ flow-control layer.
Rate limiting
Cap work to a budget per window — the emq:{q}:limiter counter spent to max, read with EchoMQ.Metrics.get_rate_limit_ttl/3.
Fairness under load
Prevent starvation — the rota serves every lane in turn, weights tilt the share.
Groups & multi-tenant fairness
Share capacity fairly across tenants — round-robin lanes with a per-group concurrency ceiling.
Batches & pipelining
Bulk enqueue in one wire flush, with a per-item verdict for each job.
Worker concurrency
The per-claim fetch ceiling and how to plan capacity.
Workshop
Rate-limit and fairly schedule codemojex's API and jobs.
How to apply
name the load problem · the technique followsThe hard part is matching the flow-control technique to the load problem you have. The four moves below are how a queue stays stable while many callers share it. Name the load problem, and the technique — and the real EchoMQ surface that becomes it — follows.
There is no single flow-control trick — only the technique that answers the load problem you have, each one a real move in EchoMQ's flow-control layer.
The workshop
echo/apps/codemojex · EchoMQ.LanesThe chapter closes with R6.06: rate-limit and fairly schedule codemojex's API and its
jobs. codemojex is a Telegram emoji-guessing game on the same stack; a flood of guesses from one room must
not starve the engine for everyone else. The budget is a per-queue counter —
emq:{q}:limiter spent down to max over its window — global across every bot
worker, because the counter lives in Valkey, not in any one process. Fairness rides the same fair-lane
ring: each room is a lane the rota serves in turn, so a busy room takes its share and no more. Bulk admits
ride EchoMQ.Jobs.enqueue_many/4, one wire flush for a burst of guesses.
The scaling layer in depth
The EchoMQ Queue pillar absorbed groups, batches, lifecycle, fairness, and the rate limit — the scaling-layer depth (the limiter wired at the claim point, fair lanes, concurrency) lives there, rung by rung.
/bcs/busFair lanes & the architecture
The manuscript bus chapter (B3) the figures are drawn from — the Valkey-native bus, the rota of fair lanes, and the BCS law that a system gates ingress on one branded namespace.
Notes on Valkey. The {q} hashtag pins every key of one queue —
emq:{q}:limiter, emq:{q}:meta, the ring, the lanes — to one of the 16384
cluster hash slots (CRC16 of the hashtag bytes, modulo 16384). One slot keeps a multi-key script legal
— no CROSSSLOT — and co-locates the limiter counter every worker spends.
valkey.io/topics/cluster-spec
Up next
R7–R8R6 keeps the queue stable under load. Each later chapter is a different concern over the same work — how the data lives in RAM, then running the tier in production.
Data Modeling
How data lives in RAM.
Production & Ops
Running the tier at scale.
References
sources · relatedSources
- Valkey — INCR — the atomic counter increment behind the fixed-window rate-limit count.
- Valkey — EXPIRE — set the window's lifetime so the budget resets when the key expires.
- Valkey — Cluster specification — the
{q}hash tag forces a queue's keys onto one of the 16384 slots. - Stripe — Scaling your API with rate limiters — the request-rate techniques a production limiter composes: token bucket, fixed window, concurrency.
Related in this course
- R6.01 · Rate limitingcap work to a budget per window — the limiter counter spent to max
- EchoMQ · the Queue pillarthe scaling layer in depth — the limiter at the claim point, fair lanes, concurrency
- The Branded Component System · the busthe Valkey-native bus and the fair-lanes architecture, manuscript B3
- R5 · Streams & Eventsthe durable, replayable log the previous chapter records