The conformance suite · dive 02

Run, and the verdict

Walk the list once, isolate each clause on its own sub-queue, clean up behind it, and answer one verdict.

run(conn, queue) walks scenarios/0 once. For each {name, contract} it derives a per-scenario sub-queue — queue <> "." <> name — runs the clause there, purges what it minted, prints one CONF line, and records pass or fail. After the loop it prints a closing tally and returns the verdict: {:ok, n} when every clause passed, {:error, failed_names} otherwise — naming exactly which clause broke. The set grows, so n is the live total, re-pinned, never a literal on the page.

The loop Sub-queues & the purge Reading the verdict Pattern & implementation References

One pass · per clause

one pass — derive a sub-queue → run the clause → purge its keyspace → emit one CONF line; repeated once per clause, then a closing tally + the verdict

One loop, one verdict

run/2 is run(conn, queue, []): with no only: option it selects every scenario of scenarios/0, and run_selected/3 is the comprehension over that selection (an only: list narrows it to the named scenarios, in the same order). Each iteration runs one clause and records a {name, ok} pair; the verdict is a fold over those pairs. A clause that raises is caught and turned into a named failure — the run never crashes, it reports.

# echo_mq — EchoMQ.Conformance
# Run the selected scenarios, each on a per-scenario sub-queue of `queue`: purge
# what it mints, print one CONF line each + a closing tally. Returns {:ok, n}
# when all pass, {:error, failed_names} otherwise. run/2 is run(conn, queue, []):
# with no only: option, select/1 picks every scenario of scenarios/0. The set
# grows by additive minor; n is the LIVE total, never hardcoded here.
def run(conn, queue, opts \\ []) when is_binary(queue) and is_list(opts) do
  selected = select(Keyword.get(opts, :only))
  # … the expect: option (:declared or :wire4), checked and held for the run
  try do
    run_selected(conn, queue, selected)
  after
    # … the previous expect: value is restored
  end
end
# …
# The loop: one pass per selected scenario, in scenarios/0 order.
defp run_selected(conn, queue, selected) do
  results =
    for {name, contract} <- selected do
      # a disjoint braced keyspace per scenario
      q = queue <> "." <> Atom.to_string(name)

      verdict =
        try do
          # drive the public surface / raw wire
          apply_scenario(name, conn, q)
        rescue
          # a raise becomes a named failure, never a crash
          e -> {:fail, Exception.message(e)}
        end
      # KEYS emq:{q}:* then DEL: leave no residue
      purge(conn, q)
      ok = verdict == :ok
      IO.puts("CONF #{name} #{if ok, do: "ok", else: "FAIL #{inspect(verdict)}"} -- #{contract}")
      {name, ok}
    end

  failed = for {name, false} <- results, do: name
  IO.puts("CONFORMANCE #{length(results) - length(failed)}/#{length(results)}")
  if failed == [], do: {:ok, length(results)}, else: {:error, failed}
end

The verdict is a fold: pass every clause and the answer is {:ok, n} for the live total; fail any and it is {:error, failed_names} — the contract names what broke.

Sub-queues & the purge

Each clause gets its own queue suffix, so the braced keyspace emq:{queue.name}: is disjoint from every other clause's. Two clauses never collide, and a failure is named, not smeared across the run. Because the queue name carries the {…} hashtag, all of a clause's keys still hash to one slot — the suite respects the same placement law the bus does.

After each clause, run/2 deletes everything under that sub-queue's prefix — KEYS emq:{q}:* then DEL the matches — and does the same for the sub-queue's dead-letter topic, emq:{q.dlq}:*, a separate slot the dead-letter scenarios park into. The suite leaves the server as it found it, so you can run it against a live system and it cleans up behind itself.

# echo_mq — EchoMQ.Conformance
# Delete everything under this sub-queue's braced prefix and its dead-letter
# topic's. The suite leaves the server as it found it.
defp purge(conn, q) do
  # Clear the scenario's OWN braced slot AND the DERIVED DLQ topic's slot
  # (emq:{q.dlq}:*, a SEPARATE hashtag slot the bus_dlq_* scenarios park into).
  # The DLQ topic is a first-class bus topic in its own braced slot, so a purge
  # of emq:{q}:* alone leaves the parked DLQ stream behind; a later run whose
  # System.unique_integer collides across BEAM restarts would then read a STALE
  # parked entry (the XRANGE COUNT-1 oldest) and mismatch the fresh MSG. Purging
  # both slots gives every DLQ scenario a clean slate (the root of L-7's flake).
  for slot <- [q, q <> ".dlq"] do
    {:ok, keys} = Connector.command(conn, ["KEYS", "emq:{" <> slot <> "}:*"])
    if keys != [], do: {:ok, _} = Connector.command(conn, ["DEL" | keys])
  end

  :ok
end

Isolation by name plus the purge is what makes the suite safe to run against a live server: no clause sees another's keys, and none leaves any behind. The braced suffix is the same hashtag discipline the whole bus uses, so the cleanup is two single-slot KEYS sweeps, the queue's keyspace and its dead-letter topic's, not a scan of the server.

Reading the verdict

The output is one CONF <name> ok -- <contract> line per clause, then CONFORMANCE <passed>/<total>. A green run ends {:ok, n}; a single failing clause yields {:error, [name]} — naming exactly which clause of the contract a port (or a regression) broke, with the rest of the run already cleaned up. Run the model below over a fixed list: an all-pass run, and one where a single clause fails.

A run · stream the CONF lines

Pick an outcome and press “run it” to stream the CONF lines.

One CONF line per clause, a closing tally, and a single verdict that either says {:ok, n} or names the clause that broke — and the run cleaned up after itself.

Pattern & implementation

the pattern (Redis Patterns Applied)

A coordination test must isolate each case and clean up after itself, or one case's residue poisons the next. Coordination teaches the atomic primitives the run asserts.

the implementation (echo_mq)

run/2 gives every clause a disjoint braced sub-queue and purges it — isolation by name, no residue, one named verdict — and a raise becomes a named failure, so the run reports rather than crashes.

The pattern asks for isolation and cleanup; the run gives both by name, and answers one verdict the contract can stand behind.

Recap — one pass, one verdict

run(conn, queue) walks scenarios/0 once, runs each clause on a disjoint braced sub-queue, purges what it mints, prints one CONF line each plus a closing tally, and returns {:ok, n} or {:error, failed_names}. The count is the live total, never a literal on the page. Next: how the set grows to a larger n without breaking the wire.

Next in the module: The additive-minor law — how a clause is added (a minor) and why changing one is a wire break (a major).

References

Where the run loop, the isolation, and the purge are grounded.

Sources

Beck — Test-Driven Development — the self-contained, self-cleaning scenario.
Valkey — KEYS — the prefix sweep the purge issues over a queue's keyspace and its dead-letter topic's.
Valkey — DEL — the cleanup that leaves no residue.
Valkey — Cluster specification — the {q} hashtag the sub-queue keeps on one slot.

Related in this course

The conformance suite — the module this dive belongs to.
The scenarios — the list run/2 walks.
The additive-minor law — how the count grows.
The Protocol — the braced keyspace each sub-queue lives in.