R6.04.1 · Batches — dive

Round-trip elimination.

N separate enqueues cost N network round-trips; one pipelined flush costs one. The win is wall-clock round-trip time — the server still runs every script, but the bytes cross the wire once. This is the whole reason a batch is faster than a loop, and it is worth seeing exactly where the saving lives so it is not confused with doing less work.

Grounding: echo/apps/echo_mq/lib/echo_mq/jobs.ex · EchoMQ.Jobs.enqueue_many/4 reduces the pairs into one EchoWire.Pipe and flushes with one exec; the per-item @enqueue script is unchanged, so the server does the same total work.

Four enqueues: a loop against a flush. Select a lane to read it. The loop pays a round-trip per item — send, wait, send, wait; the flush sends all four together and reads the replies together, one round-trip.

readout · Select a lane. The loop pays one round-trip per item — send, wait, repeat — so four enqueues cost four round-trips. The flush sends all four commands, waits once, and reads all four replies — one round-trip for the whole batch. The server runs four scripts in both.

§1

The cost of a round-trip

latency vs work

A single enqueue is a request and a reply. The client sends the command, then waits for the server to answer before it can send the next one. That wait is a round-trip: the time for the bytes to reach the server, plus the server's work, plus the time for the reply to come back. On a fast local link the network part is small; across a network boundary it dominates, and it is paid once per command.

Enqueue a thousand jobs in a loop and you pay that round-trip a thousand times, back to back — send, wait, send, wait. The server is idle for most of each cycle, waiting for the next request to arrive; the client is idle for the rest, waiting for the reply. The throughput ceiling is not the server's speed. It is the latency of the link times the number of items.

A loop pays a full request-and-reply latency per item, and the client and server take turns being idle. Under many items across a network, that latency — not the server — is the wall the throughput hits.

§2

The flush

jobs.ex · the Pipe reduce

Pipelining removes the wait. The client writes every command to the socket without pausing for the replies, then reads all the replies together. The commands still execute on the server, in order, one after another — but the client no longer pays a round-trip between each. It pays one: the time to push the whole batch out and read the whole batch back.

enqueue_many/4 is exactly this. It assembles one EVALSHA command per {id, payload} pair into an EchoWire.Pipe, then flushes the pipe once. The accumulation is a fold over the pairs; the flush is a single exec:

EchoMQ.Jobs · echo/apps/echo_mq — enqueue_many/4: reduce the pairs into one Pipe, exec onceverbatim

pipe =
  Enum.reduce(pairs, Pipe.new(conn, via: via), fn {id, payload}, pipe ->
    Pipe.command(pipe, [
      "EVALSHA",
      @enqueue.sha,
      "2",
      Keyspace.job_key(queue, id),
      Keyspace.queue_key(queue, "pending"),
      id,
      payload
    ])
  end)

with {:ok, results} <- Pipe.exec(pipe) do     ← one round-trip carries every EVALSHA

The reduce builds the command list; exec sends it and reads the replies. A thousand pairs become a thousand EVALSHA commands in one write and a thousand replies in one read — one round-trip, not a thousand. The replies return in one frame, in input order.

The fold accumulates the commands into a pipe; exec flushes them in a single round-trip and reads the replies together. N enqueues become one request and one reply.

§3

What does not change

the server still runs N scripts

The saving is on the wire, and only on the wire. Each pair still runs the full enqueue script on the server: kind-gate the id, EXISTS for a duplicate, HSET the row, ZADD the pending entry. The server does the same total work for a batch of N as for N separate enqueues — N script executions. Pipelining changes when the bytes travel, never how much the server computes.

EchoMQ.Jobs · echo/apps/echo_mq — the @enqueue script: run once per item, batch or notverbatim

if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

That is the right way to reason about it. If the server is the bottleneck — the scripts are heavy, the CPU is saturated — a batch will not help, because the batch runs the same scripts. If the round-trip is the bottleneck — many cheap writes across a network — a batch turns N round-trips into one and the throughput rises by close to the factor of N, until something else becomes the limit.

And reading the replies together does not collapse them. The flush returns a list of N results in input order, and enqueue_many/4 maps each to a verdict: 1 becomes :enqueued, 0 becomes :duplicate, an EMQKIND error becomes {:error, :kind}. So the batch buys the round-trip saving without giving up per-item visibility — every item's outcome is still there, at its own position in the list. The third dive examines that verdict model against a transaction's single result.

Where the batch helps, and where it does not. Select a dimension to read how a loop and a batch compare on it. The wire cost falls; the server work does not.

readout · Select a dimension. A batch turns N round-trips into one, but it runs the same N scripts on the server, preserves input order, and reports a result for every item — the saving is wire latency, nothing else.

Notes on ValkeyPipelining cuts the round-trip cost because the client does not wait for each reply before sending the next command; the server still parses and runs every command in order. The throughput gain is proportional to the per-command latency saved, so it is largest when the round-trip dominates the per-command server time — valkey.io/topics/pipelining.

§4

The pattern, applied

echo/apps/codemojex

the pattern

Collapse N round-trips into one flush: write the commands back to back, then read the replies together. The latency is paid once.

its EchoMQ application

EchoWire.Pipe accumulates one EVALSHA per pair and exec sends them in a single round-trip; the server runs all N @enqueue scripts, the wire carries one request and one reply.

In codemojex — a Telegram emoji-guessing game on the same stack — a single guess is admitted by Codemojex.Guesses.submit/3, one enqueue per guess. When a burst of player commands arrives together, sending each as its own round-trip is the slow path; the flat-queue batch shape admits the burst in one flush over emq:{cm}:pending, so the wire cost is paid once for the whole burst rather than once per guess.

Driving that batch into the consumer at scale — the pool that flushes it, the claim that drains it — is the queue's scaling layer, taught in the EchoMQ course.

/echomq/queue

References

sources · related

Sources

Valkey — Pipelining — the canonical description of why pipelining cuts round-trip time without reducing server work.
Valkey — EVALSHA — the per-pair command the pipe accumulates; one cached script run per item.
Valkey — ZADD — the score-0 pending insert each enqueue script runs on the server, batch or not.

Related in this course

R6.04 · Batches & pipeliningthe module hub
R6.04.2 · Chunking across a poolflushing the batch across N pooled connectors
R6.04.3 · Partial-failure handlingreading the per-item verdicts the flush returns
/echomq/queuethe batch driven into the consumer
/bcs/busPart B3, the Valkey-native bus the figures draw from