jonnify·bcs /bcs/cache/cache-aside/one-fill-per-herd

B4.1 · dive 2 — one fill per herd

Two hundred misses, one load.

Misses route through the table's owner, and concurrent misses on one key coalesce onto a single flight — the first caller's flight checks L2, falls through to the loader, writes both layers, and the owner replies to every waiter with the one answer. The drill is the gate: 200 concurrent cold readers, loader runs 1, coalesced waiters 199, every reader holding the one answer. The hit path that makes the cache worth having is priced on the same record: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire.

Source: content/bcs4.1.md, quoting bcs_rung_4_1_check.out; the module is committed at runtimes/elixir/lib/echo_store/table.ex.

The herd coalescer. The derive line's counterfactual against the law's outcome, both verbatim from the record; the waiter split is computed live from the committed counts.
without single-flight 200 loads — one per reader under the law loader runs 1 — and 199 coalesced waiters share the one answer
readout · Select an outcome. The counterfactual is the record's own derive line; the law's outcome is gate E3, frozen in §1 below.
§1

The transcript

bcs_rung_4_1_check.out · verbatim · this dive reads the herd and speed lines
content/echo_data/runtimes/elixir/bcs_rung_4_1_check.out · the slice this dive readsfrozen
…
derive (herd): 200 concurrent cold readers without single-flight run 200 loads; the law demands the misses coalesce onto one flight -- expect loader runs 1 and 199 coalesced waiters
E3 herd ok -- the thundering herd survived with one fill: 200 concurrent cold readers, loader runs 1, coalesced waiters 199, every reader holding the one answer
derive (speed): a hit is a caller-side lookup on a public read-concurrency set plus the kind gate and a counter bump -- expect 250,000 to 1,500,000 hit reads per second on this core; an L2 GET pays a loopback round trip, and Appendix A committed 29,456 sequential round trips per second, near 34 us each -- expect the L1 hit at least 10 times cheaper than the wire
E4 speed ok -- measured: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire -- the L1 hit is 40 times cheaper than the round trip it replaces, inside the derived band
…
PASS 6/6

The elided lines hold the header, E1–E2, and the jitter and bound drills — dive 1 and dive 3 read them, and the module hub freezes the record whole. The derive lines are part of the record: each measurement is bounded before it is taken, and the committed line then lands inside its own band.

§2

The flight

content/bcs4.1.md · What · Decisions

The first caller's flight checks L2, falls through to the loader, writes both layers, and the owner replies to every waiter with the one answer. The pattern has a name in the Go world — singleflight, "a duplicate function call suppression mechanism" — and the Go port's fill discipline already has its idiom: golang.org/x/sync/singleflight is one-fill-per-herd as a library.

The decision: flights are processes, not owner code. A slow loader blocks its own flight, never the owner — other keys keep filling, puts keep landing. A crashed flight is a monitored event: every waiter gets a typed error, nobody hangs. And loader errors are not cached — a broken upstream surfaces immediately instead of being memorized; negative caching is a deliberate omission, revisitable under its own gate.

Two hundred simultaneous cold readers at the open are the normal case in this domain, and the gate says they cost one load. The manuscript plans the referee chapter — B4.5 · The Cache Referee — to hold the comparison set to this drill.

§3

The priced pair

content/bcs4.1.md · Hits at ETS speed · Boundaries

A hit is a caller-side lookup against a public, read-concurrency ETS table, plus the kind gate and one atomic counter bump. The read path never enters the owner's process — the owner is consulted only when there is owner's work to do; that is the whole architecture. Derived first, then measured: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire -- the L1 hit is 40 times cheaper than the round trip it replaces, inside the derived band.

This is what makes 762 ns a hit — always read beside its pair, the 31 us the wire charges — and it is why stats ride :counters instead of GenServer.call. The boundary travels with the number: one core, loopback, this container — the ratios travel, the absolutes describe this machine.

The hit against the wire. The two committed costs on one nanosecond axis — the bar widths are computed live from the record's pair. Select a view of the same two numbers.
L1 hit · 762 ns a caller-side lookup, the kind gate, a counter bump L2 GET · 31 us a loopback round trip on the same wire
readout · Select a view. One pair, never split: 762 ns per L1 hit against 31 us per L2 GET — the committed line calls the hit 40 times cheaper.

Static record — the pair and its derivation: 1311621 hit reads per second (762 ns each) · 31 us per L2 GET · the derive line's band 250,000 to 1,500,000 hit reads per second on this core · Appendix A's floor 29,456 sequential round trips per second, near 34 us each.

§4

References

sources · related

Sources

Related