Uber: resilience & staggered sharding.

Docstore is Uber’s distributed database. A read against it is durable and correct but pays the full database cost; at Uber’s scale, repeating identical reads against Docstore is expensive in both latency and load. Uber’s answer was an integrated Redis cache — a layer in front of Docstore that the application reads through transparently. On a hit the value comes back from Redis in memory; on a miss the read falls through to Docstore and the result populates the cache.

The wins were the cache-aside wins at scale: lower latency, less database load, lower cost. A documented use case ran at over 6 million reads per second with a 99% cache hit rate. The title figure is the aggregate: 40 million reads per second served from the integrated cache.

But a cache in front of a database changes the failure model. When the cache is healthy it absorbs the load; when the cache is sick, the traffic it was absorbing lands somewhere — and where it lands decides whether a cache problem stays a cache problem or becomes a database outage. The rest of the module is the three things Uber did so the second never happened.

Staggered sharding — misalign the two sharding schemes. Uber sharded the Redis cluster using a scheme different from the database sharding scheme. The reason is the failure model: if Redis and Docstore were sharded the same way, a single Redis cluster going down would dump all of its missed traffic onto the one Docstore shard that backed it — a hot shard, a correlated failure that can take the database shard down too. By deliberately misaligning the schemes, a single Redis cluster’s load spreads across multiple database shards instead of concentrating on one. Misalignment turns a correlated failure into a spread one.

The sliding-window circuit breaker. When a Redis node is down, every get and set routed to it pays a latency penalty before it fails — and at millions of requests per second, that penalty compounds. Uber added a sliding-window circuit breaker per node: count errors per node in each time bucket, sum the counts over the window’s width, and once the error count crosses a threshold, short-circuit a fraction of the requests to that node. If errors keep accumulating, the breaker trips: no requests go to the node at all until the window passes and the counts age out. The circuit breaker is the canonical resilience pattern — fail fast against a known-bad dependency rather than wait for each call to time out.

Graceful degradation. A Redis miss or a short-circuited request falls through to Docstore. The read is slower than a cache hit, but it is served — degraded, not failed. The combined effect was measured: P75 latency down about 75%, P99.9 latency down over 67%, and latency spikes limited. The same use case at over 6M reads/second failed over successfully to a remote region when its primary region had trouble — the cache going away in one region degraded to the remote path rather than taking the read down.

Uber’s integrated cache is the lens; the system the reader is building is EchoMQ, backed by Valkey. The honest framing is same goals, different mechanisms. Uber’s three techniques aim at three goals: do not concentrate a failure (staggered sharding), do not keep calling a sick dependency (the circuit breaker), and stay available when a dependency is gone (graceful degradation). The EchoMQ connector reaches the same three goals by different means.

One guard holds the parallel straight. The connector’s max_pending → :overloaded bound is backpressure / a bounded in-flight queue, not a circuit breaker. A circuit breaker tracks a dependency’s error rate over a window and stops calling it; the max_pending bound caps how many requests can be in flight at once and refuses the overflow. They share a goal — do not pile work onto a struggling path — but the mechanisms differ: backpressure is a depth limit, a circuit breaker is an error-rate trip. The circuit-breaker concept in this module is Uber’s; echo_mq’s analogues are backpressure, fail-fast, and reconnect-backoff.

The worked consumer is codemojex. Every guess goes through EchoMQ.Connector, so a guess inherits these properties for free: if the wire is saturated the caller gets a fast :overloaded rather than an unbounded queue, and if Valkey drops the socket the in-flight guess fails :disconnected (a fast, honest error the caller can retry) while the connector reconnects with backoff. The game stays responsive under trouble instead of hanging on a sick wire.

The module follows the resilience arc — place the failure, then break the circuit, then degrade gracefully. Read them in order.